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1. Introduction 

In this paper we study semiclassical quantizations of Poincare maps arising in scattering 
problems with hyperbohc classical flows which have topologically one dimensional trapped 
sets. The main application is the proof of a fractal Weyl upper bound for the number of 
resonances/scattering poles in small domains. 

The reduction of open scattering problems with hyperbolic classical flows to quantizations 
of open maps has been recently described by the authors in [29]. In this introduction we 
show how the main result of the current paper applies to the case of scattering by several 
convex obstacles, and explain ideas of the proof in that particular setting. 



Let O = Uj^i Oj where Oj d are open, strictly convex, have smooth boundaries, 
and satisfy the Ikawa condition: 

(1.1) Ofe n convex hull(Oj U O^) = , j^k^i. 

The classical flow on (M" \ C^) x §"^^ (with the second factor responsible for the direction) 
is defined by free motion away from the obstacles, and normal reflection on the obstacles - 
see the figure above and also §6.3 for a precise definition. An important dynamical object 
is the trapped set, K, consisting of {x,C,) G (K" \0) x S""-*^ which do not escape to infinity 
under forward or backward flow. 

The high frequency waves on \ O are given as solutions of the Helmholtz equation 
with Dirichlet boundary conditions: 

(-A - X^)u = 0, ue if^(M" \ O) n iJo^(M" \0), XeR. 
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The scattering resonances are defined as poles of the meromorphic continuation of 

it:(A) = (-A-A^)-^ : LLmp(K" \ o) ^ i:L(K" \ c) 
to the complex plane for n odd and to the logarithmic plane when n is even. 
The multiplicity of a (non-zero) resonance A is given by 

(1.2) mniX) = rank J) R{C)dC , 7 : t ^ A + ee^'^'* , < t < 1 , 0<e<l. 

Our general result for hyperbolic quantum monodromy operators (see Theorem 4 in §5.4) 
leads to the following result for scattering by several convex obstacles. 

Theorem 1. Let O — IJj=i ^ union of strictly convex smooth obstacles satisfying 

(1.1). Then for any fixed a > 0, 

(1.3) Yl ^«(A) = 0(r^+°) , r^oo, 

— a<Im A 
r<\X\<r+l 

where 2^ + 1 is the box dimension of the trapped set. 

If the trapped set is of pure dimension (see ^5.4) then the bound is O(r^). In the case of 
n — 2 the trapped set is always of pure dimension, which is its Hausdorff dimension. 

We should stress that even a weaker bound, 

m«(A) = 0(r'^+^+°) , 

— a<Im A 
l<|A|<r 

corresponding the standard Weyl estimate 0{r'^) for frequencies of a bounded domain, was 
not known previously. Despite various positive indications which will be described below 
no lower bound is known in this setting. 

The study of counting of scattering resonances was initiated in physics by Regge [36] and 
in mathematics by Melrose [26] who proved a global bound in odd dimensions: 

Y ruRiX) = 0(r") . 

|A|<r 

This bound is optimal for the sphere and for obstacles with certain elliptic trapped trajec- 
tories but the existence of a general lower bound remains open - see [46] , [47] and references 
given there. For even dimensions an analogous bound was established by Vodev [54] 

The fractal bound (1.3) for obstacles was predicted by the second author in [39] where 
fractal upper bounds for the number of resonances were established for a wide class of 
semiclassical operators with analytic coefficients (such as — /i^A -|- V with V equal to a re- 
striction of a suitable holomorphic function in a complex neighbourhood of M"); promising 
numerics were obtained in [23] for the case of a three bump potential, and in [37] for the 
Henon-Heiles Hamiltonian. The bound of the type (1.3) was first proved for resonances 
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Figure 1. This figure, taken from [24], sliows resonances computed using 
tlie semiclassical zeta function for scattering by three symmetrically placed 
discs of radii a and distances r between centers. The horizontal axis rep- 
resents 2a/7o where 70 is a classical rate of decay (the imaginary parts of 
resonances tend to cluster at ImA ~ — 7o/2 — see [24]) and the vertical axis 
corresponds to the best fit for the slope of log A^(a,r)/logr, that is the ex- 
ponent in the fractal Weyl law (1.4). The three lines correspond to the three 
values of the Hausdorff dimension /i. 



associated to hyperbolic quotients by Schottky groups without parabolic elements (that is 
for the zeros of the Selberg zeta functions) in [15], and for a general class of semiclassical 
problems in [45]. Theorem 4 below provides a new proof of the result in [45] in the case 
of topologically one dimensional trapped sets. The new proof is simpler by avoiding the 
complicated second microlocalization procedure of [45, §5]. The reduction to Poincare sec- 
tions obtained using the Schrodinger propagator [29] replaces that step. The only rigorous 
fractal lower bound was obtained in a special toy model of open quantum map in [30] . For 
some classes of hyperbolic surfaces lower bounds involving the dimension were obtained in 
[19]. 

In the case of three discs in the plane, results of numerical experiments based on semi- 
classical zeta function calculations [12] were presented in [24]. They suggest that a global 
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version of the Weyl law might be vahd: 

(1.4) Ar(Q;,r)= mniX) ^ C{a)r''+\ r ^ oo , 

ImA>-a, |A|<r 

see Fig. 1. A similar study for the scattering by four hard spheres centered on a tetrahedron 
in three dimensions was recently conducted in [8] , and lead to a reasonable agreement with 
the above fractal Weyl law, at least for a large enough. We stress however that the method 
of calculation based on the zeta function, although widely accepted in the physics literature, 
does not have a rigorous justification and may well be inaccurate. Experimental validity 
of the fractal Weyl laws is now investigated in the setting of microwave cavities [22] — see 
Fig. 2. The theoretical model is precisely the one for which Theorem 1 holds. The fractal 
Weyl law has been considered (and numerically checked) for various open chaotic quantum 
maps like the open kicked rotator [38] and the open baker's map [30, 33]. Theorems 2 and 
3 below lead to a rigorous fractal Weyl upper bound in this setting of open quantum maps 
with a hyperbolic trapped set. Fractal Weyl laws have also been proposed in other types of 
chaotic scattering systems, like dielectric cavities [55], as well as for resonances associated 
with classical dynamical systems [49, 5, 9]. 

The proof of Theorem 1 uses Theorem 4 below, which holds for general hyperbolic quan- 
tum monodromy operators defined in §2. Here we will sketch how these operators appear 
in the framework of scattering by several convex bodies. In the case of two obstacles they 
were already used in the precise study of resonances conducted by Ch. Gerard [13]. The 
detailed analysis will be presented in §6. 

To connect this setting with the general semiclassical point of view, we write 

and consider the problem 

P{z)u = , P{z) = (j^{-h^A -l)-z^ u{x) , u\ao= , u outgoing. 

The precise meaning of "outgoing" will be recalled in §6. This rescahng means that inves- 
tigating resonances in {r < j Aj < r + 1, Im A > —a}, corresponds to investigating the poles 
of the meromorphic continuation of P{z)~^ in a fixed size neighbourhood of (in these 
notations the resonances are situated on the half-plane Re^; > 0). 

The study of this meromorphic continuation can be reduced to the boundary through 
the following well-known construction. To each obstacle Oj we associate a Poisson operator 
Hj{z) : C^{dOj) C°°(R" \ Oj) defined by 



P{z)Hj{z)v{x) = , xeR'^\Oj, Hj{z)v\aoj= v , Hj{z)v outgoing. 
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Figure 2. The experimental set-up of the Marburg quantum chaos group 
http://www.physik.uni-marburg.de for the five disc, symmetry reduced, 
system. The hard walls correspond to the Dirichlet boundary condition, that 
is to odd solutions (by reflection) of the full problem. The absorbing barrier, 
which produces negligible reflection at the considered range of frequencies, 
models escape to infinity. 



Besides, let 7^- : C°°(]R") — )■ C°°{dOj) be the restriction operators, 'yju =^ u laoy As 
described in detail in §6.1 the study of the resolvent P{z)~^ can be reduced to the study of 

J J 
{I-M{z,h))-' : 0C-(9O,)^0C-(9O,), 

where 

(1.5) (A^(..ft)),= {-"f« Ity. 

The structure of the operators •jiHj^z) is quite complicated due to diffractive phenomena. 
In the semi classical/large frequency regime and for complex values of z, the operators Hj{z) 
have been analysed by Gerard [13, Appendix] (Im^; > —C) and by Stefanov-Vodev [48, 
Appendix] (Imz > —C\og{l/h)). We refer to these papers and to [17, Chapter 24] and 
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[27] for more information about propagation of singularities for boundary value problems 
and for more references. 

Using the propagation of singularities results obtained from the parametrix (sec §6.2) 
the issue of invertibility of (/ — Ai{z, h)) can be microlocalized to a neighbourhood of the 
trapped set, where the structure of A4{z, h) is described using /i- Fourier integral operators 
— see §3.4 for the definition of these objects. 

At the classical level, the reduction of the flow to the boundary of the obstacles proceeds 
using the standard construction of the billiard map on the reduced phase space 

J 

B*dO=\_\ B*dOj , 

i=i 

where B*dOk is the co-ball bundle over the boundary of Ok defined using the induced 
Euclidean metric (see Fig. 3 in the two dimensional case). Strictly speaking, the billiard 
map is not defined on the whole reduced phase space (e.g. in Fig. 3 it is not defined on 
the point p'); one can describe it as a symplectic relation F C B*dO x B*dO, union of the 
relations Fij C B*dOi x B*dOj, i ^ j encoding the trajectory segments going from Oj to 
Of. 

{p',p) e Fij c B*dOi X B*dOj 

3 t > , C e §""^ , xedOj, x + t^e dOi , {uj{x),0 > , 
{ui{x + tC),0 <0, = P, 7ri(x + tC, = P' ■ 
(here tt^ : SgQ^{W^) — >■ B*dOk is the natural orthogonal projection.) 
To the relation F we can associate various trapped sets: 

oo 

(1.7) r± = Pi F^\B*do) , r = r+ n T- . 

fe=0 

Notice that T is directly connected with the trapped set K for the scattering flow, deflned 
in the beginning of this introduction: 

r n B*dOj = 7ij{K n s*9a^ {W)) . 

The sets 7± and T for the 2D scattering problem of Fig. 3 arc shown in Fig. 4. 

The strict convexity of the obstacles entails that the trapped rays are uniformly unstable: 
in the dynamical systems terminology, T is an invariant hyperbolic set for the relation F 
(see §2). 

The boundary of B*dOk, which consists of covectors of length one, corresponds to rays 
which are tangent to dOk (the glancing rays), and these produce complicated effects in 
the operators Hj{z). However, Ikawa's condition (1.1) guarantees that none of these rays 
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Figure 3. (after [32]) Reduction on the boundary for the symmetric three 
disk scattering problem (the distance D between the centers is 3, the radius of 
the disks is 1). Left: the trajectories hitting the obstacles can be parametrized 
by position of the impact along the circles dOi, (length coordinate G 
[0, 27r)) and the angle between the velocity after impact and the tangent to the 
circle (momentum coordinate cos(p G [—1, 1]). We show three short periodic 
orbits, and a transient orbit. Right: reduced phase space B*dO = L\^^iB*dOi 
of the obstacles (we concatenate the three length coordinates into s G [0, Gvr)). 
Each of the 3 periodic orbits is represented by 2 points on the horizontal axis 
(circles, crosses, squares), while the transient orbit is represented by the 
successive points p, p' = F{p). Vertical lines delimit the partial phase space 
used in Fig. 4. 
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Figure 4. The unstable manifold, 7+, the trapped set, T, and the stable 
manifold 71 and for the system of Fig. 3. Compared with the notations of 
Fig. 3, the horizontal axis is given by the union of three segments of dOj, 
parametrized by a single coordinate s = Sj/vr + (i — l)/3, Sj G [0,7r/3], 
i = 1,2,3 (this partial phase space is delimited by the vertical lines in Fig. 3, 
right). The sets 7± are smooth along the unstable/stable direction and fractal 
transversely, while T has the structure of the product of two Cantor sets. The 
left strip corresponds to the one indicated in Fig. 3. (the figures are from 
[32]). 



belongs to the trapped set: 

(1.8) rn5*9O = 0. 

A the quantum level, the operator Ai{z,h) has the properties of a Fourier integral 
operator associated with the billiard map away from the glancing rays, but its structure 
near these rays is more complicated due to diffraction effects. Fortunately, the property 

(1.8) implies that, as far as the poles of (/ — Ai{z, h))~^ are concerned, these annoying 
glancing rays are irrelevant. Indeed, we will show in §6 that the operator Ai{z, h) can be 
replaced by a truncated operator of the form 

(1.9) M{z, h) = n,e-^" M{z, h) e^'" 11^ + OL^dO)-,L^^9o){h''))llu , 

where g{x,C,) is an appropriate escape function, and Uh is an orthogonal projector of finite 
rank comparable with h^~^, microlocally corresponding to a compact neighbourhood of T 
not containing the glancing rays. The operator M{z, h) is thus "nice" : it is of the same type 
as the hyperbolic quantum monodromy operators constructed in [29] to study semiclassical 
scattering problems such as —h'^A + V{x), with V G C^(]R"). The study of the poles of 
(/ — M{z, h))~^ can then be pursued in parallel for both types of problems (see §5). 

For simplicity let us assume that the trapped set T is of pure dimension dimT = 2^. 
The fractal Weyl upper bound (1.3) corresponds to the statement that, for any fixed r > 0, 
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there exists > such that 

(1.10) — ^tr / {I - M{z,h))-^d,M{z,h)dz <Crh-'' . 

27rZ J{\z\=r} 

To prove it we want to further modify the monodromy operator M{z, h), and replace it by 

(1.11) M{z,h) = Uw{e-^^M{z,h)e^^ + 0{e))Uw , 0<e<l. 

Here G is a finer escape function, and Uw is a finer finite rank projector, now associated with 
a much thinner neighbourhood of T, of diameter ~ h^, and therefore of volume comparable 
to /j,2(2("-i)-2/i) (^ggg ^jig definition of the dimension in (5.22) below). This projector, and the 
escape function G, will be constructed in §5 using symbols in an exotic pseudodifferential 
class, and the associated symbol calculus. The rank of an operator microlocalized to a set 
in T*]R"^^ of volume v is estimated using the uncertainty principle by 0{h~"''^^v). Hence, 
when V ~ /j,2(2("-i)-2/i) -^g obtain the bound 0{h~'^) for the rank of Hw- This in turn 
shows that 

log I det(7 - M{z, h))\ = Oih-^") . 

We also show that at some fixed point Zq, the above expression is bounded from below by 
—Ch~^. Jensen's inequality then leads to (1.10). 

In the case where T is not of pure dimension, one just needs to replace /i by /i+O in the 
above estimates. 

The paper is organized as follows. In §2 we give a general definition of hyperbolic open 
quantum maps and quantum monodromy operators. This definition will be compatible with 
both the case of several convex obstacles, and the case of semiclassical potential scattering, 
after the reduction of [29]. The technical preliminaries are given in §3 where various facts 
about semiclassical microlocal analysis are presented. In particular, we investigate the 
properties of exotic symbol classes, which will be necessary to analyze the weight (escape 
function) G used to conjugate the monodromy operator in (1.11), and to construct the 
projection Uw- The weight G is constructed in §4, using our dynamical assumptions. 
In §5 we construct the projection Uw, and prove the resulting fractal upper bounds on 
the number of resonances in the general framework of hyperbolic quantum monodromy 
operators. Finally, §6 is devoted to a detailed study of the obstacle scattering problem: 
we show that the reduction to the boundary operator Ai{z), combined with propagation 
of singularities, leads to a hyperbolic quantum monodromy operator (1.9), to which the 
results of the earlier sections can be applied. 

2. Hyperbolic open quantum maps 

In this section we start from a hyperbolic open map, and quantize it into an open quantum 
map. The quantum monodromy operators constructed in [29] have the same structure, but 
also holomorphically depend on a complex parameter z. 
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Let Yj R*^, j = 1, • • • J, be open contractible sets, and let 

Y l]Yj G [_\r' , 

j=i j=i 

be their disjoint union. We also define a local phase space, 

./ J 

U = \_\UjC\_\ T*^^ , Uj open, Uj ^ T*Yj . 
j=i j=i 

Let T W be a compact subset and suppose that 
is an invertible transformation which satisfies 

(2.1) f G F dU xU , (/is identified with its graph), 

where F is a smooth Lagrangian relation with boundary, and dF fl / = 0. We assume that 
F is locally the restriction of a smooth symplectomorphism (see below for a more precise 
statement). In particular, F is at most single valued: 

{p',p),ip",p)eF =^ p' = p\ 

and similarly for F~^ . By a slight abuse of notation, we will sometimes replace the graph 
notation, (p',p) G F, by the map notation, p' = F{p). As a result of (2.1), for any p & T 
the tangent map dfp : TpU — > Tf^p^U is well defined, and so is its inverse. 

Such a relation F can be considered as an open canonical transformation on U. Here, 
open means that the map F {F~^) is only defined on a subset 71r{F) {71l{F), respectively) 
of W; the complement U\TrR{F) can be thought of as a hole through which particles escape 
to infinity. 

We now make a stringent dynamical assumption on the transformation /, by assuming 
it to be hyperbolic (equivalently, one says that T is a hyperbolic set for F). This means 
that, at every p eT, the tangent space TpU decomposes as 

i) TpU = Ep © E~ , dim F^ = d , with the properties 

(2.2) it) dfp{Ef) = Ef^^^ , 

Hi) 3 0<e <1, yveEj , Vn > 0, Wdf^'^iv)]] < C^"||t;|| . 

This decomposition is assumed to be continuous in p, and the parameters 9, C can be 
chosen independent of p. It is then a standard fact that [20, §6. 4, §19.1] 

iv) T 3 p I — 7- E^ C Tp{U) is Holder-continuous 

(2.3) v) any p E T admits local stable(— )/unstable(-|-) manifolds W^^{p), 

tangent to E^ 
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Let US now describe the relation F more precisely. It is a disjoint union of symplecto- 
morphisms 

Fik '■ Afe C Uk — > Fik{Dik) — C Ui , 
where Aik C Ui and Dik <zUk are open neighbourhoods of the arrival and departure subsets 

of r, 

Aik^^{peTi : r\p)^Tk}^rif\f{Tk), Tj^^TnUj, 

Afe = {p e Tk : /(p) G ?-} = Tfc n f-\%) . 

(sec Fig. 6 for a plot of the sets Aik and Da- for the scattering by three disks, and Fig. 4, 
center, for the trapped set). We also write 

j k 

and similarly for the other sets above. Notice that we have D — 7rij(F), A — t:l{F). 

On the quantum level we associate to F hyperbolic open quantum maps defined as follows: 

Definition 2.1. A hyperbolic open quantum map M — M{h), is an h-Fourier integral 
operator quantizing a smooth Lagrangian relation F of the type described above, 

MeIo+{YxY,F'), 

(here F' is the twisting of the relation F so that F' becomes Lagrangian in T*{Y x y) - 
see ^3.4 for the definition of this class of operators). In particular, M is microlocalized in 
the interior of F' : its semiclassical wavefront set (see (3.25) j satisfies 

WFh(M) n 9F' = . 

We also assume that there exists some qm £ C^{T*Y), with supp qm contained in a compact 
neighbourhood yVofT,yV<^ ttrIF), and auip) = 1 in a smaller neighbourhood W' ofT, 
such that 

(2.4) M(7 - Am) = C»l2-,l2(/i^°) , Am = Op^aM) , 

with Nq ^ 1, independent of h. This means that M{h) is very small, microlocally outside 
W. 

Assume Ti^ is an orthogonal projector of finite dimension comparable with h^'^, with Hh 
equal to the identity microlocally near W. We will then also call open quantum map (or 
truncated open quantum map ) the finite rank operator 

M{h) = IlhM{h) Uh = M{h) + Ol^^lAH''') ■ 

A hyperbolic quantum monodromy operator is a family of hyperbolic open quantum maps 
{M{z, h)} (or their finite rank version M{z, h) ) associated with the same relation F, which 
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depend holomorphically on z in 

(2.5) n = n{h) = [-R{h),Ri]+i[-Ri,Ri], R{h) ^-^oo, Ri >0 , 
as operators ^ L"^ . Furthermore, there exists a decay rate Tm > such that 

(2.6) \\M{z,h)\\ KCe^^'^""', h < ho, z e Q{h) . 

We will also consider truncated monodromy operator M{z, h). The cutoff' Am, the projector 
Uh and the estimates (2.4) are assumed uniform with respect to z E Q{h). 

This long definition is taylored to include the monodromy operator M{z, h) constructed 
for open hyperbolic flows with topologically one dimensional trapped sets, through a 
Grushin reduction — see [29]. That construction shows that the exponent A^o in (2-4) can 
be taken arbitrary large, and that we can take domain in (2.5) with R{h) — Clog(l//i), 
for some C > 0, and Ri large but fixed. 

For the scattering by several convex bodies described in the introduction, we will be able 
(using propagation of singularities) to transform the boundary operator Ai{z,h) of (1.5) 

into a monodromy operator M{z, h) of the form above, with arbitrary A^o, see §6. 

3. Preliminaries 

The general preliminary material and notation for this paper are the same as in [29, §2]. 
We specifically present properties of exotic symbols and weights necessary to construct 
the escape function G and the projector Ilw in (1.11), and analyze their interaction with 
Fourier integral operators. Some of the material is taken directly from [29, §2], [45, §3], 
and some developed specifically for our needs. 

3.1. Semiclassical pseudodifFerential calculus. We recall the following class of symbols 
on T*W^ (here m. A; e R, 5 e [0, 1/2]): 

S^^k{T*W^) = |a e C°°(T*R'^ x (0, 1]) : Va, /3 G N", 

where we use the standard notation (•) = (1 + •^)^/^. 

The Weyl quantization a'^{x, hD) of such a symbol is defined as follows: for any wave- 
function u in the Schwartz space <S(R'^), 

a'"u{x) = a'"{x,hD)u{x) = [0^l{a)u\{x) 

see [7, Chapter 7] for a detailed discussion of semiclassical quantization, and [41, Appendix], 
[10, Appendix D.2] for the semiclassical calculus for the symbol classes given above, and its 
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implementation on manifolds. When 5 = 0orm = A; = 0we will generally omit to indicate 
those indices. We denote by *^'''(M'^), ^''"''=(R'^), or ^'(R'^) the corresponding classes of 
pseudodifferential operators. 

For a given symbol a G S{T*M.'^) we follow [43] and say that its essential support is 
contained a given compact set K d T*]R'^, 

ess-supp^ acK T^M*^ , 

if and only if 

Vx e S{T*R'^) , suppx n is: = =^ xa e h°°S{T*R'^) . 

Here S denotes the Schwartz space. The essential support of a is then the intersection of 
all such K^s. 

For A e *(M'^), A = Op";; (a), we call 

(3.2) WF,,(A) = ess-supp;, a 

the semiclassical wavefront set of A. (In this paper we are concerned with a purely semi- 
classical theory and will only need to deal with compact subsets of T*]R'^.) 

Let u = u{h), \\u{h)\\i^2 = 0(h^^) (for some fixed N) be a wavefunction microlocalized in 
a compact set in T*W^, in the sense that for some x G C^(T*M'^), one has u = x^u+Os{h'^). 
The semiclassical wavefront set of u is then defined as: 

(3.3) WF h{u) = {:{{x,0 eT*R'^ : 3aeS{T*R'^), a{x,0 = l, Wa"" u\\l2 = 0{h°^)} . 
For future reference we record the following simple consequence of this definition: 

Lemma 3.1. If u{h) — Oi,2^^n^{h~^) is a wavefunction microlocalized in a compact subset 
ofT*W and 

v{h){x')=u{h){Q,x'), {xi,x') gM" 

then v{h) = OL2(Rn){h~^~^^'^) and v{h) is microlocalized in a compact subset o/T*M"~-'-. 
In addition we have 

(3.4) WF;,(^;) C {{x',C') G T*R^-' : 3^1 G M, (0, x', Ci, O e WF^«)} . 

We point out that unlike in the case of classical wave front sets we do not have to make 
the assumption that ^ when {x',0,^i,^') G WFh{u). 

Proof Let x e C~(T*M") such that u = x'"u + Os{h'^) - it exists by the assumption that 
X is microlocalized in a compact set. 

By choosing G C°^(]R) such that iP{Cl) = 1 ^oi (x,^) G suppx we have ip{hDxi)u — 
u + OsihP^) and we can simply replace u by ip{hDxi). Then 

2 
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where Th is the unitary semiclassical Fourier transform (see [10, Chapter 2]). Integrating 
in x' gives the bound ||t'||L2(]Rn-i) = 0{h~^~^/'^). 

Similar arguments prove the remaining statements in the lemma. □ 
Semiclassical Sobolev spaces, Hf^{X) arc defined using the norms 

, , = ll(^ - h^^^,^y'M\L\m^) , ^ = K\ 

(3.5) 

— ~ ^^^g) '^||l2(x) ) ^ a compact manifold, 
for any choice of Riemmanian metric g. 

3.2. Si spaces with two parameters. We now refine the symbol classes ST''^, by in- 

2 _ 2 

troducing a second small parameter, h e (0, 1], independent of h. Following [45, §3.3] we 
define the symbol classes: 

(3.6) a e 5r""''(T*M'^) ^ |5:5|'a(x,0l <C'«^/i-'"/i-'^(|[)'^'"™(e)'~'^', 

where in the notation we suppress the dependence of a on /i and h. When working on R*^ 
or in fixed local coordinates, we will use the simpler classes 

(3.7) aeSiT*^^) <^ |9"a|<Ca, a e ^i(r*R'') <^ l^'^a] < Ca(/i//i)^'"' ■ 
We denote the corresponding classes of operators by vI'"'''"''^(]R'^) or ^i. 

2 2 

We recall [45, Lemma 3.6] which provides explicit error estimates on remainders in the 
product formula: 

Lemma 3.2. Suppose that a,b&Si, and that — o . Then for any integer N > 
we expand 

^ I fih V 

k=0 ' ^ ^ 

The remainder ejsf is bounded as follows: for some integer M independent of N, 
(3-9) Yl {h'^d^,,^,y,,)f{ta{D)/2f^'d'^^a{x,0d"%y,v) 

ai+a2=a r^K^xT'IRd ^eN4d ,|/3|<M 



where 



a{D) = a{D^, D^; Dy, D,) ""^ {D^, Dy) - {D„ D,) , 



is the symplectic form on T*M x T 
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Notice that, due to the growth of the derivatives of a, 6, the expression (3.8) is really an 
expansion in powers of h rather than h. On the other hand, if a e {T*W^) and b is in 
the more regular class S {T*W^) , then 

k=0 

We also recall [45, Lemma 3.5] which is an easy adaptation of the semiclassical Beals's 
lemma — see [7, Chapter 8] and [10, §9.5]. Because of a small modification of the statement 
we present the reduction to the standard case: 

Lemma 3.3. Suppose that A : 5(R'^) 5'(R'^). Then A = Op)^(a) with a e Si if 
and only if, for any N > and any sequence {ij}jLi of smooth functions which are linear 
outside a compact subset of T*M.'^, 

(3.10) II adop5j'(£i) o • • • adop^(^;v) ^^*l|L2(Md) < C'^^^^^^''^lkllL2(Rd) , 
for any u e S{R'^). 

Proof. We use the standard rescaling to eliminate h: 

{x,o = {h/h)Hx,o, 

and implement it through the following unitary operator on L^(M'^): 

(3.11) U^/~^uix) = {h/h)iu{{h/h)h) = {h/h)iu{x) . 
One can easily check that 

Op^a) = U^lj^Opl{d)U,/~, , where a(x, f) = aiih/h)kx, f)) . 

Notice that the symbol a G S{T*R'^) if a G Si{T*R'^). In the rescaled coordinates, the 
condition (3.10) concerns /i-pseudodifferential operators: it reads 

(3.12) II adop^,(4) o • • • o adop^(^~^) Opf (5)^11^2 < C^^||'u||i2 . 
Let us prove the statement in the case where the £/s are linear: then 

are also hnear, and Beals's lemma for /i-pseudodifferential operators [7, Prop. 8.3] states 
that (3.12) for any N is equivalent with d e S{1). 

We finally want to show that, ii d e Si, then (3.12) holds for ^j's which are compactly 
supported. Actually, for £j e S{1) we may use Lemma 3.2 to see that 

adop»(£^) Op'^id) = OL2^L<h^h^) , 

which implies (3.10). □ 



16 S. NONNENMACHER, J. SJOSTRAND, AND M. ZWORSKI 

The rescahng (3.11) can be used to obtain analogues of other standard results. Here is 
one which we will need below. 

Lemma 3.4. Suppose a & Si and that suprp^-^a \a\ > c> 0, with c independent of h and h. 
Then 

||Opna)|U2_i2<sup|a|+e»(/i). 

Proof. We first apply the rescaling (3.11), so that Op^(a) is unitarily equivalent to Op^(a), 
where d e 5'(T*M'^), and sup \ d\ — sup \ a\. We then note that 

Opf (a-)* Opf (a) = Opf (lan + OL.^r.<h) , 

and that the sharp Carding inequality (see for instance [7, 7.12] or [10, Theorem 4.21]) 
shows that 

(sup|a|)2-0pf(a)*0p^(a)>-C^, 
from which the lemma follows. □ 

3.3. Exponentiation and quantization. As in [45], we will need to consider operators 
of the form exp(G"^(x, /iD)), where G e 5?'°'"°°(T*M'^). To understand the properties of 

2 

the conjugated operators, 

exp(-G'"'(a;, HD)) P cxp(G'^'(^> hD)) , 

we will use a special case of a result of Bony and Chemin [1, Theoreme 6.4] — see [45, 
Appendix] or [10, §9.6] 

To state it we need to recall a more general class of pseudodifferential operators defined 
using order functions. A function m : T*W^ — )■ R+ is an order function in the sense of [7] 
iff for some N >0 and C > 0, we have 

(3.13) Vp, p' e r*R<^, m{p) < Cm{p'){p - p')^ . 

The class of symbols corresponding to m, denoted by S{m), is defined as 

a G S{m) ^ \d^d^a{x,0\ < Ca^m{x,0 

(in this notation S{1) is the class of symbols we had called S{T*'R'^)). If mi and m2 are 
order functions in the sense of (3.13), and aj G S{mj) then (we put h = 1 here), 

a^(x, D) a^(a;, D) = U"{x, D) , be S{mim2) , 
with b given by the usual formula 
6(x,0 = ai ti a2{x,^) 

(3.14) 

= exp{ia{D^i,D^i;D^2,D^2)/2) ai{x\i^)a2{x'^,i^)\xi=x2=x,e=e=^ ■ 
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Note that here we do not have a small parameter h, so aijia2 cannot be expanded as a power 
series. The value of the following proposition lies in the calculus based on order functions. 
A special case of [1, Theoreme 6.4], see [45, Appendix], gives 



Proposition 3.5. Let m be an order function in the sense of (3.13); and suppose that 

g e C°^(T*M'^;M) satisfies, uniformly m {x,^) e T*R'^, 



The estimates on Bt G S{rn}) depend only on the constants in (3.15) and in (3.13). In 
particular they are independent of the support of g. 

Since m* is the order function exp(tlogm(a;,^)), we can say that, on the level of order 
functions, quantization commutes with exponentiation. 

This proposition will be used after applying the rescaling (3.11) to the above formalism. 
For the class Si, the order functions are defined by demanding that for some N >0, 



(3.15) g{x,0-logm{x,0^O{l), d^d'^g{x,0 ^ 0{1) , 



a\ + \P\>l. 



Then for any i e 



(3.16) exp(i^"'(x, D)) = B^{x, D) , Bt e S{m^) . 

Here ex.p{tg'^{x, D))u, u e <S(R'^), is constructed by solving 

dtu(t) = g'^ix, D)u{t) , 'u(O) = u . 



2 



(3.17) 




The corresponding class is defined by 




2 



We will consider order functions satisfying 




satisfies 



(3.20) 




a\>l. 



Using the rescahng (3.11), we see that Proposition 3.5 implies that 
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For future reference we also note the following fact: for A e ^^(R'^), 

(3.22) A - e-G'"(-''^^) A e^^'C-.'^^) = h'^i^-^'^h'^ a^x, hD) , a^^S._. 

The following lemma will also be useful when applying these weights to Fourier integral 
operators. 

Lemma 3.6. Let U T*M'^, and let x G C^{U). Take Gi, G2 two weight functions as in 
(3.19); such that 

{G,-G2)\u^0. 

Then, 

^Gr{x,hD) ^ ^G^{x,hD) ^ Os'-^s(h^) . 

Proof. We just give the proof of the first identity, the second being very similar. Let us 
differentiate the operator e~*'^2'(^.'»^) ^tGY{x,hD)^w. 

^^-tG^{x,hD) ^tGfix,hD) ^ ^-tG^{x,hD)i^QW _ qw-j ^tGY{x,hD) 

For each t e [0,1], the operator ^ \ii i(^rn,\) is bounded on L^, and has its 

semiclassical wavefront set contained in suppx. Due to the support property of Gi — G2, 
we get 

(G^ - G^) e*^5"(=^.'»o) ^ Os'^s{h^) , 
and the same estimate holds once we apply e~**^2 (^.'^-D) on the left. As a result, 

^-G^{x,hD) ^Gf(x,hD) _ = Os'^sih^) , 

from which the statement follows. □ 

3.4. Fourier integral operators. We now follow [10, Chapter 10], [43] and review some 
aspects of the theory of semiclassical Fourier integral operators. Since we will deal with 
operators in \E'i, the rescaling to /i-semiclassical calculus, as in the proof of Lemma 3.3, 
involves dealing with large /i-dependent sets. It is then convenient to have a global point 
view, which involves suitable extensions of locally defined canonical transformations and 
relations. 

3.4.1. Local symplectomorphisms. We start with a simple fact about symplectomorphisms 
of open sets in T*M.'^. 

Proposition 3.7. Let Uq and Ui be open neighbourhoods of (0,0) and k. : Uq ^ Ui a 

symplectomorphism such that fi;(0,0) = (0,0). Suppose also that Uq is star shaped with 
respect to the origin, that is, if p & Uq, then tp G Uq for < t < 1. 

Then there exists a continuous, piecewise smooth family 

{i^t}o<t<i 



FRACTAL WEYL LAW FOR OPEN QUANTUM CHAOTIC MAPS 19 

of symplectomorphisms Kf : Uq ^ Ut — Kt{Uo), such that 

(i) «;t(0, 0) = (0, 0) , 0<t<l, 
^3 23) Ki — K, kq — id[7o > 

(iii) = {i^t)*Hq^ , 0<t<l, 

where {qt}o<t<i is a continuous, piecewise smooth family ofC°°{Uo) functions. 

The last condition means that Kt is generated by the (time dependent) Hamiltonian 
vector field Hg^ associated with the Hamiltonian qt^. 

Sketch of the proof: Since the group of linear symplectic transformations is connected, we 
only need to deform k to a linear transformation. That is done by taking 

kt{p) = f^itp)/t, te[o,i], 

which requires the condition that Uq is star shaped. It satisfies Rq = dK,{0,0). □ 

As a consequence we have the possibility to globalize a locally defined symplectomor- 
phism: 

Proposition 3.8. Let Uq and Ui be open precompact sets in T*M'' and let k : Uq ^ Ui be 
a symplectomorphism which extends to Uq 2) JJq, an open star shaped set. Then k extends 
to a symplectomorphism 

k : T*W^^T*W^, 

which is equal to the identity outside of a compact set. k can be deformed to the identity 
with qt's in (iii) of (3.23) supported in a fixed compact set. 

Proof. Let k^ be the extension of k to Uq. By Proposition 3.7 we can deform k^ to the 
identity, with {d/dt)k1 = {k!l)^Hq^, qt G C°°(t/o). If we replace qt by qt — XQt where 
X G C^{Uq), and x = 1 in C/q, the family of symplectomorphisms of T*M.'^ generated by 
{qt)o<t<i satisfies 

'^t\uQ= f^tluo , /«o = id, i^i\uo= 1^1 ^tl~CLf(,^ ^^C[7(, • 

Then k = Ki provides the desired extension of and the family Kt the deformation with 
compactly supported g^'s. □ 

The proposition means that, as long as we have some geometric freedom in extending our 
symplectomorphisms, we can consider local symplectomorphisms as restrictions of global 
ones which are isotopic to the identity with compactly supported Hamiltonians. We denote 
the latter class by /C: 

K={k : T*W^ T*R'^ , (3.23) holds with compactly supported qt.} . 



^This generation can also be expressed through the more usual form ^ = Hg/^ , where q't = qt° k^^. 
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We note that, except for d = 1, 2, it is not known whether every k which is equal to the 
identity outside a compact set is in /C. 

For K G /C we now define a class of (semiclassical) Fourier integral operators associated 
with the graph of k,. It fits in the Heisenberg picture of quantum mechanics - see [51, §8.1] 
for a microfocal version and [10, §§10.1,10.2] for a detailed presentation on the semiclassical 
setting. 

Definition 3.9. Let k e K as above, and let < S < 1/2. The operator U : <S(R'^) 
5'(M'^) belongs to the class of h- Fourier integral operators 

Is(R'xR',C'), C'^{(x,C,y,-v) ■■ (x,0^<y,v)}, 

if and only if there exists Uq G ^^(R'^), such that U — U{\), where 

(3.24) hDtU{t) + U{t)Q{t) = {}, Q(t) = Op]^(gt) , t/(0) = C/q . 

Here the time dependent Hamiltonian qt € C^(T*M°') satisfies (iii) of (3.23). 
We will write 

Io+{m'^ X R'^, C) = Pi IsiM'^ X R^ C) . 

5>0 

For A e /o+(R'' X R*^, C) we define its /i-wavefront set 

(3.25) WF,(A) {{x, e; -V) e C : (y, 77) G WF,(C/o)} C T^R'^ x T*R'' , 

where WF/j([/o) is defined in (3.2). 

We recall Egorov's theorem in this setting — see [H-Sl, Appendix a] or [10, Theorem 
10.10]. 

Proposition 3.10. Suppose that U = U{1) for (3.24) with Uq = I, and that A = Op^^ia), 
aeS{T*R'^). Then 

U-^AU = B, 5 = 0pn&), b-K*aeh^S{T*R'^). 
An analogous result holds for a G S'™'''^(T*R'^). 

More generally, ifT& hO^'^ x R'^, C") and A as above, then 

AT ^TB + TiBi , 

where B = Op)^(6) is as above, Ti G /^(R'^ x R'', C), and Bi G h^-^^'$s(^'^)- 

Remark. The additional term TiBi is necessary, due to the fact that T may not be elliptic. 

The main result of this subsection is an extension of this Egorov property to operators 
A in more exotic symbol classes. 
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Proposition 3.11. Suppose that k, E JC and T e /^(R'^ x R*^; C), where C is the graph of 
K. Take A = e^'"(^''^^)Op;;'(ao); where Uq G Si{1) and G satisfies (3.20). Then 

AT ^TB + TiBi , 

(3.26) B = e('^*«)'"(^''^^)0pn6o) , - ^*ao G /iUi^|(l) , 

5i = /i5(i-25)^iOpn&i), heSi{e^'^), Ti e /^(R'^ X R^ C") ■ 

The proof is based on two lemmas. The first one is essentially Proposition 3.11 with 
G = 1, 5 = 0, and T invertible: 

Lemma 3.12. Suppose U = U{1), where U{t) solves (3.24) with U{0) = I. Then for 
A^Op^{a),aeS^{l), 

(3.27) U~^AU = B, 5 = 0pn&), b-K*aeh^h^Si{l). 

Proof. We will use the variant of Beals's lemma given in Lemma 3.3 above. Let ij be 
as in (3.10), j — 1, - ■ ■ ,N, and denote 

op-,{i,{t)) = mopWj)u{t)-'- 

We first claim that ij{t) are, to leading order, linear outside of a compact set: 

(3.28) i,{t) = {K;'yi, + h\, rteS{l). 
To prove it, we see that the evolution (3.24) gives 

(3.29) hDtijitr = [Q{t)Jj{tr] Lj{t) , 
where 

Q(t) U(t)OpMU(t)-' = Op;:(g* + Osmih')) , qt = (K^'Tqt . 
Then, using Lemma 3.2, 

L,it)-ih/i)Op-,{HMt))^h'S{l). 
Since {d/dt)Kt = Hg^, (3.29) becomes 

dt{K;£j{t)) = Osa){h') , 

which implies (3.28). 

For A in the statement of the lemma let us define A{t) as 

A{t) = U{t)-'AU{t) , hDtA{t) = [Qit),A{t)] , A{0) = A . 
We want to show that A{t) — Op'^{a{t)) where a{t) e Si. We will prove that using (3.10): 

(3.30) adop»(£i) o • • • adop^ie^) A{t) = U (t)"^ (adop-(£i(t)) o • • • adop»(^^(t)) A) U (t) . 
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In view of (3.28) and the assumptions on A we see that the right hand side is 
and thus A{t) = Op;;'(a(i)), a{t) e ^i. One has B = A{1). 

1 ~ 3 

We now need to show that a{t) — K^a e h^h^Si. For that define 

A{t) := Opn<a). 
As in the proof of Egorov's theorem, we calculate 

hD,A{t) = %t(^f^^;a^=%-{H,Xta) 

= yOp-({?t,<a}) = [Q{t),A{t)] + hlhlE{t), 

and Lemma 3.2 shows that E{t) — Op'^{e{t)), where e{t) & Si. A calculation then shows 
that 

hDt{U{t)A{t)U{ty^) = U{t)E{t)U{t)-^ , 

and consequently, 

(3.31) A{t) - A{t) ^ ih'^h'^ [ U{t)-^U{s)E{s)U{s)-^U{t)ds. 

Jo 

We have already shown that conjugation by U{s) preserves the class ^i, hence the integral 
above is in Si. Then A{t) - A{t) e h^h^Si. □ 



1 ~ 3 

Remark. The use of Weyl quantization is essential for getting an error of size h^h^. To 
see this wc take f/ to be a mctaplectic transformation - see for instance [7, Appendix to 
Chapter 7]. The rescaling (3.11) simply changes it to the same metaplectic tranformation, 
U, with h as the new Planck constant. Then 

U-^d'"(x, hDi)U = {k*af (x, hDs) , 

that is we have no error term. Had we used right quantization, a{x, hD), we would have 
acquired error terms of size h, which could not be eliminated after rescaling back. For the 
invariance of the Si calculus see [56, §3.3]. 

The arguments of the previous lemma can be extended to encompass the weight function 
G of (3.19), which may not be in (indeed, G may be unbounded), but which has bounded 
derivatives. 
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Lemma 3.13. Let U be as in Lemma 3.12. For G satisfying (3.20) we define 

g^ = U-^G'"{x,hD)U : S^S. 

Then 

gi = G'^{x,hD), Gi- K*G eh^USi{l). 



Proof. Since U : S ^ S, the operator Qi maps S to S. We now proceed as in the proof of 
Lemma 3.12, noting that 

e {h/h)-\'^\'^'^Si , \a\>l, uniformly for t e [0, 1] . 

Lemma 3.2 shows that only terms involving derivatives appear in the expansions, hence 
the same arguments apply. □ 

We now combine these various lemmas. 

Proof of Proposition 3.11: Suppose that T G hiR'^ x R'^,C'), meaning that T = UqU, 
where U = U{1) satisfies (3.24) with ^7(0) = /. By Egorov we may write T = UUi, with 
Ui e "i/s- Then, in the notation of Lemma 3.13, 

AT = U{U-^AU)Ui = Uexp GiU-^AqUUi = TB + UB^ , 

where, using Lemma 3.13, 

B = expGiU-^AoU = exp{{K*G)'")Bo , BqE^i, 

and, using (3.22), 

B, = [exp{{K*Gr), U,]Bo + exp((«*G')-)[So, U^] 

= exp{{K*Gr) (C/i - exp{-{K*G)'")Ui exp{{K*G)'") + [Bo, C/i]) 
e /il(i-25)/^i exp((K*G)"')*i . 

□ 



3.4.2. Lagrangian relations. We are now ready to extend the above semiglobal theory 

("semi" because of our special class of symplectomorphisms /C) into a construction of 
/i-Fouricr integral operators associated with an arbitrary smooth Lagrangian relation on 
F C T*Y X T*Y (as introduced in §2). This construction will naturally be done by splitting 
F into local symplectomorphisms defined on (small) star shaped sets. 

Eventually, we want consider the full setup of §2, that is taking F the disjoin union of 
Fij C Ui X Uj, defining our Fourier integral operator T e I^^Y x Y,F') as a matrix of 
operators, 

T = (Ti,)i<i,,<j , T,j e h{Yi X F,-, Fl^) , 
and finally take h+iY x F, F') = U5>o Io+{Y x Y, F'). 
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To avoid too cumbersome notations, we will omit the indices and consider a single 
Lagrangian relation F (Zlf xU, where U m T*Y m T*^^, U' m T*Y' C T*W^ two open 
sets, and define the classes UiY' x Y). 

Fix some small £ > 0. On U we introduce two open covers of U , 

L 

e=i 

such that each is star shaped around one of its points, and has a diameter < e. We also 
introduce a smooth partition of unity {xe)i=i,...,L associated with the cover (C/^): 

(3.32) J2xeip) = l, peneigh([/), xe & C^{Ue,[OA]) ■ 

e 

F can be seen as a canonical map defined on the departure subset t:r{F) C C/, with range 
7rL(F) C U' . Let us call F^ — F\^^ its restriction to U^. 

The set of interior indices I such that C/^ C t^r{F) will be denoted by C. 

For each interior index ^, the symplectomorphism is the extension of = F \u^, so 
we may apply Proposition 3.7, and produce a global symplectomorphism G /C, which 
coincides with F^ on the set Ui. The previous section provides the family of /i- Fourier 
integral operators /^(M"' x M'',C^), where is the graph of Hi. For each interior index ^ 
we consider a Fourier integral operator e /^(M'^ x W^, C[), and use the partition of unity 
(3.32) to define 

Due to the support properties of Xii the operator is actually associated with the restric- 
tion Fi of Hi on Ui- The sum 

rpR def rp^ 

defines a Fourier integral operator on M'^, microlocalized inside til{F) x 71r{F), which we 
call IsiW^ X R'^,F'). 

Finally, since we want the wavef unctions to be defined on the open sets Y, Y' rather than 
on the whole of R'^, we use cutoffs e C^{Y, [0, 1]), * e C^{Y, [0, 1]) such that *(x) = 1 
on 7r(C/), *'(a;) = 1 on ^([/'). 

We will say that T : V'{Y) C~(F') belongs to the class 

h{Y'xY,F') 

iff 

m'Tm ^m'T^m for some e {W^ x M'^, F') , 

and 

T- ^'T^ = C(/i~) : P'(F) ^C°^(F'). 
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We notice that 7rR(WF/j(T)) is automatically contained in the support of J^^g^ Xt-, a strict 
subset of 7rii{F): in this sense, the above definition of Is{Y'xY, F') depends on the partition 
of unity (3.32). However, for any subset oi W F , one can always choose a cover (t/^) 
such that \Jj(zcUt D TiniW). In particular, the assumption Tn dF = we made in §2 
shows that such a subset W may contain the trapped set T. 

3.4.3. Conjugating global Fourier integral operators by weights. By linearity one can gener- 
alize Prop. 3.11 to a Fourier integral operator T G Io+{Y' x Y, F'), and by linearity to the 
full setup of §2. The observable and weight G are now functions on T*Y' or on T*]R'^. 

If G is supported inside iiLiF) C T*Y', then F*G = G o F is a smooth function on 
nR{F), which can be smoothly extended (by zero) outside. In each U£ we apply Prop. 3.11 
to T^, and obtain on the right hand side terms of the form 

e^**^ Opn&o) +TiBi, Ti e /^(M'' x R^, F'), e h^/^-^h^/^^i{e^*'^) , 

and WF;,(Ti) C WF^(T). 

Since tt ji{WF ^{Ti)) d Ui, Lemma 3.6 shows that only the part of k^G inside is relevant 
to the above operator, that is a part where K£ = F. Therefore we have 

This proves the generalization of Prop. 3.11 to the setting of the relation F C T*Y' x T*Y, 
in case suppG C t^l{F). 

In case G is not supported on t:l{F), the notation e^^**^)"" still makes sense microlocally 
inside t^r{F). Indeed, take X-,X ^ ^T{'^l{F)), X = 1 near 7ri:(WF/j(T')), x = 1 near suppx- 
Lemma 3.6, with V — 'Kl{F) implies that 

e«"'r = e^'"r + C»(0, G^'^xG. 
The above generalization of Prop. 3.11 then shows that 

The weight F*G is only relevant on 7r/j(WF/i(T)) nR{F), so it makes sense to write the 
first term on the above right hand side as 

emphasizing that this operator does not depend (modulo Os'-^s{h°°)) the way we have 
truncated G into G. 

For the same reason, the symbol class 5*1 (e^**^) makes sense if we assume that the symbols 
are essentially supported inside ttrIF). We have just proved the following generalization 
of Prop. 3.11: 
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Proposition 3.14. Take F a Lagrangian relation as described in ^2, and T e Is{Y x 
Y,F') as defined above. Take G G C'^{T*Y) a weight function satisfying (3.20); and A — 
gG™(a;,/iD)Qptt)^Q^-j^ ^jJiq^-q ^/jg symbol Go e 'S'i(l); ess-suppao <^ 'Kl{F)- 

Then the following Egorov property holds: 

AT^TB + TiBi , 

(3.33) S = e(^*«)^(-''^^)OpnM , bo-F*aoeh'^hlSiil), ess-supp bo m nn{F) , 
Bi = h'^^^-^^^UOp^{bi) , 6i e Si{e^*^) , Ti e Is{Y x Y,F') . 

This proposition is the main result of our prehminary section on exotic symbols and 
weights. Our task in the next section will be to construct an explicit weight G, adapted to 
the hyperbolic Lagrangian relation F. 

4. Construction of escape functions 

Escape functions are used to conjugate our quantum map (or monodromy operator), so 
that the conjugated operator has nicer microlocal properties than the original one, even 
though it has the same spectrum. More precisely, an escape function G{x,^) should have 
the property to strictly increase along the dynamics, away from the trapped set. An escape 
function g{x, ^) has already been used in to construct the monodromy operators associated 
with the scattering problems in [29] : its effect was indeed to damp the monodromy operator 
by a factor ~ outside a fixed neighbourhood of T. Our aim in this section is to construct 
a more refined escape function, the role of which is to damp the monodromy operator 
outside a semiclassically small neighbourhood of T, namely an /i^/^-neighbourhood. For 
this aim, it is necessary to use the calculus on symbol classes 5*1 (m) we have presented in 
§3. 

Our construction will be made in two steps: first in the vicinity the trapped set T, 
following [45, §7] (where it was partly based on [39, §5]), and then away from the trapped 
set, following an adaptation of the arguments of [14, Appendix]. For the case of relations 
F as in §2 which arise from Poincare maps of smooth fiows, we could alternatively use the 
flow escape functions given in [45, Proposition 7.7]. However, the general presentation for 
open hyperbolic maps is simpler than that for fiows, and will also apply to the monodromy 
operators obtained from the broken geodesic flow of the obstacle scattering problem. 

4.1. Regularized escape function near the trapped set. Let 7± be the outgoing and 
incoming tails given by (1.7) in the case of the obstacle scattering. For an open map F 
with properties described in §2, these sets are deflned by 



r± = {p : F^"(p) e U, Vn>0}, 
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where F^" = o • • • o denotes the usual composition of relations. We note that 
7± are closed subsets of U, and due to the hyperbolicity of the flow, they are unions of 
unstable/stable manifolds H^^(p), p & T. 

Remcirk Before entering the construction, let us consider the simplest model of hyperbolic 

map, namely the linear dilation (x,^) h- )■ (Ax, A~^^) on T*]R, with A > 1. In that case, the 
trapped set is reduced to one point, the origin, and the sets 7± coincide with the position 
and momentum axes. In this case, a simple escape function is given by 

(4.1) Gix, 0=x'-e = d{p^ T-f - d{p, nf , 

and it can be used (after some modification) to analyse scattering flows with a single 
hyperbolic periodic orbit [13, 14]. 

The construction of G(,x, ^) in the case of a more complex, but still hyperbolic, trapped 
set, inspires itself from the expression (4.1) [39]. Our first Lemma is a construction of two 
functions related with, respectively, the outgoing and incoming tails. It is a straightforward 
adaptation of [45, Prop. 7.4]. For a moment we will use a small parameter e > 0, which 
will eventually be taken equal to h/h. 

Lemma 4.1. Let V he a small neighbourhood of T and F : V — )■ FiV) be the symplecto- 
morphic restriction of F. Then, there exists Co > and a neighbourhood V <^ V of the 
trapped set, such that the following holds. 

For any small e > there exist functions (p± e C°°(V U -F(V); [e, oo)) such that 
ip±{p) ~ d{p,T±f + e, 

± {^±{P) - ^±{F{p))) + Coe lp±{p) , peV, 
5"^±(p) = 0(^±(p)'-l"l/'), 

Here and below, a ^ b means that there exists a constant C > 1 ( independendent of e) such 
that b/C <a<Cb. 

To prove this lemma we need two preliminary results. 



Lemma 4.2. Suppose T C R"* is a closed set. 
such that 

(4.3) ip>e, ip^e + d{;rf, 

where the estimates are uniform on W^. 



For any e > there exists (p e C°°(M™), 

= e»((^i-i"i/2) , 



Proof. For reader's convenience we recall the proof (sec [45, Lemma 7.2]) based on a Whit- 
ney covering argument (see [16, Example 1.4.8, Lemma 1.4.9]). For 6 <^ 1 choose a maximal 
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sequence Xj e R"* \ F such that d{xj, Xi) > 5d{xi, T) (here d is the Euchdean distance). We 
claim that 

(4.4) \jB{x^,d{x^,V)/%)^W^\V. 

3 

In fact, if X is not in the sequence then, for some j, d{x,Xj) < 6d{xj,r) or d{x,Xj) < 
6d{x,r). In the first case x G B{xj, d{xj, r)/8) if 5 < 1/8. In the second case, d{x,Xj) < 
5{d{xj,T) + d{x,Xj)) which means that x e B{xj,d{xj,T)/8), if 5/(1 — 5) < 1/8. Hence 
(4.4) holds provided (5 < 1/9. 

We now claim that every x G M"^\r lies in at most Nq = iVo((5, m) balls B{xj, d{xj, r)/2). 
To see this consider x and i ^ j such that d{x,Xj) < d{xj,r)/2 and d{x,Xi) < d{xj,V) /2. 
Then simple applications of the triangle inequality show that 

d{xi, Xj) > 26d{x, r) /3 , d{xj, T) < d{x, F) /2 . 

Hence 

B{xi, 5d{x, r) /3) n B{xj, 6d{x, T) /3) = , 

B{xe, Sd{x, r) /3) c B{x, 4:d{x, T) /3) , i, j . 

Comparison of volumes shows that the maximal number of such i's is (4/5)"*. 

Let X e C^iR""; [0, 1]) be supported in B{0, 1/4), and be identically one in B{0, 1/8). 
We define 

a(Xj,L )>y/e 

We first note that the number non-zero terms in the sum is uniformly bounded by Nq. In 
fact, d{xj, r) + < 2d{xj, F), and hence if x((x — Xj)/{d{xj, T) + y^)) ^ then 

1/4 > \x-Xj\/{d{xj,V) + ^e) > {l/2)\x- Xj\/d{xj,T), 

and X e B{xj, d{xj,d{xj, T))/2). This shows that (pe{x) < 2iVo(e + d{x, F)^), and 

d'^ip.ix) = 0{{d{x, F)2 + e)^-l"l/2) , 

uniformly on compact sets. 

To see the lower bound on ^p^ we first consider the case when d{x, F) < C^/e . 

^eix)>e>ie + dix,T)^)/C'. 

If d{x, F) > Cy/e then for at least one j, x((a; — Xj)/{d{xj, F) + a/c)) = 1 (since the balls 
B{xj, d{xj, F)/8) cover the complement of F, and xit) = 1 if \t\ < 1/8). Thus 

ip,{x) > e + d{xj, F)2 > (e + d{x, F)2)/C , 

which concludes the proof. □ 

The second preliminary result is essentially standard in the dynamical systems literature, 
resulting from the hyperbolicity of the map / on T- 
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Lemma 4.3. For V a small enough neighbourhood of T , there exist < 6'i < 1 and C > 
such that, for any K >Q and p e V such that F^{p) remains in V for all < k < K, we 
have 

d{F\p),T+)<CeU{p.T+), 0<k<K. 
The same property holds in the backwards evolution with 7+ replaced by 71 . 

Proof. We want to use the fact that the map F is strictly contracting in the direction 
transerse to 7+ (unstable manifold). To state this contractivity it is convenient to choose 
coordinate charts adapted to the dynamics, and containing the points F^{p). Let us assume 
that V is an ^-neighbourhood of T, with £ > small. By assumption, each pk = F'^(po), 
Po = P, lies in an £-size neighbourhood of some & T, k < K. As a result, the sequence 
(Mfc) satisfies d{F{Mk), M^^i) < Ce: it is a Cs-pseudoorbit. Prom the shadowing lemma 
[20, §18.1], there exists an associated orbit 

TV, = F{Nk-i) e T, d{Nk,Mk) < 5, 

with 6 small if Ce is small. Hence, d{Nk, Pfc) < 5 + e. 

Besides, the distance d{pQ, 7+) is equal to the distance between po and a certain local 
unstable leaf Wi'^^{Po), with Pq e TnV. We will consider the point Qo = W{^^{No) D 
W^^{Po) e T and its images Qk — F{Qo) to construct our coordinate charts (it'^, s'^), such 
that the local stable and unstable manifolds W^^{Qk) (see [20, §6.2]) are given by 

w{;,iQk) = {(0,.'=)} , 1^+ (Qfc) = {(«^o)} . 

From the uniform transversality of stable/unstable manifolds, these coordinates can be 
chosen such that for the Euchdean norm we have 

\\u II + ||s II ~ d[Qk,P ) , 
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uniformly ior < k < K. We also have \\u''\\, \\s'^\\ < C{e + 6). See Fig. 5 for a schematic 
representation. In this coordinates, the point pk — {u^, s'') is mapped into 

(4.5) F(u\ s'') = (Aku" + ak{u\ s'), + Pk{u\ s")) 

with ak, smooth functions, ak{0,s) = I3k{u,0) = 0, dak{0,0) = d(3k{0,0) = 0, and the 
contraction property H^^^H < i' < 1- 

This contraction implies that 

\\s''\\ < {u + C{5 + eyf\\s^\\, 0<k<K. 

Wc can choose e, 6 small enough such that 6i =^ i/+C(5+£) < 1. Finally, d{pk, W^ioc(Qfe)) ~ 
1 1 5*^11 satisfies 

d{pk,r+)<d{pk,w+,{Qk))<c9'id{po,r+). 

□ 

Proof of Lemma We adapt the proof of [45, Proposition 7.4] to the setting of a 

discrete dynamical system. Let (p± be the functions provided by Lemma 4.2, respectively 
for r = 7±. As above we call F the restriction of F on V, and similarly call F~^ — F~^\y. 
For > 1 to be determined below, we consider the following neighbourhood of T: 

K+l ^ ^ 

(4.6) V= Pi F\V), and define V = V U F(V) . 

k=-K-l 

In words, V is the set of points p e V, whose orbit remains in V in the time interval 
[—K — 1, K + 1]. The following functions are then well-defined on V: 

k=0 

Lemma 4.3 shows that if V (and therefore V) is small enough, then there exist 9i e (0, 1) 
and C > 1, such that 

(4.7) d{F^\p),T±) < C9'ld{p, r±) , 0<k<K. 
It thus follows that 

(4.8) ip± ~ ip±ip) ~ d{p,r±f + e, 

with implicit constants independent of K. This establishes the first statement in (4.2). To 
obtain the second statement we see that, for any p G V, 

^+(p) - ^+(F(p)) = ^+(p) - ^+(F^'+i(p)) 

^_(F(p)) - ^_(p) = ^-{F{P)) - ^-(F-^'ip)) . 
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In view of (4.8), we find 

Mp) - MF^^\P)) = Mp) + 0{d{F^^\p), + e) 

= Mp) + o{e'^d{p,T,)' + e) 

and similarly for (/?_. Taking K large enough so that 0{6i) < 1/2, we obtain, for some 
Co > 0, the required estimates: 

±(^±(p) - ^±(^(p)) + Coe - (p±{p) . 

The estimate 

a"^±(p) = 0(^±(p)i-H/2), 
follows from the properties of (p± stated in Lemma 4.2. It remains to show that 

(4.9) lp+{p) + ip_{p)^d{p,Ty + e. 

This results [39, 45] from the uniform transversahty of the stable and unstable manifolds 
(near the trapped set). Indeed, this transversahty implies that, for any two nearby points 
Pi, P- e T and p near them, we have 

(4.10) d(p, w,tM n w,-My ~ d(p, w,tMT + d(p, w,-M)' ■ 

Besides, since 7+ (71) is a union of local unstable (stable, respectively) manifolds, for any 
p near T the distance d{p,T±) is equal to d{p,W{^^{p±)) for some nearby points p±. We 
thus get 

(4.11) d{p, + d{p, T-f ~ rf(p, (p+) n w{-^^{p^)f > d{p, rf . 

On the other hand, d{p, T) = d{p, po) for some po G T, so that 

d{p,rr = dip,w,^M^w^jpo)r 

- d{p, W^^MT + d{p, W^-Mf > d{p, r+) + d{p, T-) . 
We have thus proven the transversahty result 

(4.12) d{p,Tf ^d{p,T+f + d{p,T-f , pnee^rT. 

The statement (4.9) then directly follows from (4.8). □ 

From the properties of Lemma 4.1, and in view of the model (4.1), it seems tempting to 
take the escape function of the form (p+{p) — (p^{p). Yet, since we want to be an order 
function, G cannot grow too fast at infinity. For this reason, following [45, §7] we use a 
logarithmic flattening to construct our escape function: 

Lemma 4.4. Let (p± he the functions given by Lemma 4-i- For some M ^ 1 independent 
of e, let us define the function 

(4.13) G = log(Me + ^_) -log(Me + ^+) 
on the neighbourhood V of the trapped set defined in Eq. (4.6). 
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Then there exists Ci > such that 

G = 0(log(l/e)), 9^"G = (9(mm(^+,^_)-^) = 0(e-^), |«| > 1 , 

(4.14) a;(G(F(p))-G(p)) = 0(min(^+,^_)-^) = 0(e-^), |a|>0, peV, 

peV, d{p, Tf > Cie =^ G{F{p)) - G{p) > 1/Ci . 

Proof. Only the last property in (4.14) needs to be checked, the others following directly 
from Lemma 4.1. For this aim we compute 

(4.15) G{Fip)) - dip) = log (l + ^-(^(/^))- f-(/^) ) + log (l + MPl^lliM) . 
^ ^ ^ ^ ^ ^ V Me + ^4p) ) ^\ Me + ^+{F{p)) ^ 

Using (4.12), the condition d{T,pf > C^e implies that d(T+,pf > Cae or d(T-,pf > Cae, 
and C2 can be taken as large as we wish if C\ is chosen large enough. 

Let us take care of the first term in (4.15). For this we need to bound from below the 
ratio 

(4.16) i?-(p)'^'^P^t|#. 

Let us call C3 > 1 a uniform constant for the equivalences in (4.2). The second equivalence 
shows that 

^_(F(p)) - ^_(p) > ^_(p)/C3 - Coe. 

Since the function x i->- is increasing for x > 0, the ratio (4.16) satisfies R-{p) > 

—Cq/M. If we take M large enough, we ensure that 

log(l + i?_(p))>-2Co/M. 

Furthermore, in the region where d{p,T-Y > C2e, the first statement in (4.2) shows that 
^-(p) > ^, so that 

If we take Ci (and thus C2) large enough, C4 is nonnegative, and log(l + R^{p)) > C4/2. 
By increasing M and Ci if necessary, we can assume that C4 > GCq/M. 

The same inequalities hold for the second term in (4.15) (the condition is now d{p, 7+)^ > 
Cse). 

Finally, if d{p, T)^ > Cie, we find the inequality 

G{F{p)) - G{p) > ^ - ^ > ^ . 
Increasing Ci if necessary, the right hand side is > 1/Ci. □ 
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4.2. Final construction of the escape function. We now set up the escape function 
away from the trapped set. We recall that D is the departure, resp. arrival sets of the open 
relation F. 

Lemma 4.5. Let W2 he an arbitrary small neighbourhood of the trapped set, and W3 (s D 
large enough (in particular, we require that W3 3) suppaM, where aM is the function in 
Definition 2.1). Then, there exists go G C^{T*Y), and a neighbourhood Wi d W2 of the 
trapped set, with the following properties: 



In [29] such a function go was obtained as the restriction (on the Poincare section) of an 
escape function for the scattering flow, the latter being constructed in [14, Appendix]. 

The proof of this Lemma for a general open map satisfying the assumptions of §2 will 
be given in the appendix.lt is an adaptation of the construction of an escape function near 
the outgoing tail performed in [6] and [53] . 

Now we want to glue our escape function G constructed in Lemma 4.4, defined in the 
small neighbourhood V of T, with the escape function g^ defined away from the trapped 
set: the final escape function G will be a globally defined function on T*Y. One crucial 
thing is to check that this function is the logarithm of an order function for the Si class 
— see (3.18) and (3.20). 

The following construction is directly inspired by [45, Prop. 7.7] 

Proposition 4.6. Let V, G be as in Lemma 4-4) (^"^d choose the neighbourhoods Wi such 
that r (E (E V (E W3 (E t^r{F). 

Take x G C^O^') equal to 1 in >V2 U F(W2) <^ V. Construct an escape function go as in 
Lemma 4-5, and define 



Then, provided C5 is chosen large enough, the function G satisfies the following estimates: 



(4.17) 



VpeWi, go{p)=0, 
ypeWs, go{F{p))-go{p)>0, 
Vp e Ws \ W2, go{F{p)) - go{p) > 1 • 



G xG + G, log(l/e) go e (T*F). 



(4.18) 



|G(p)|<C6log(l/e), = 0(6-H/2) , |«|>1, 

PGW2 ^ G{F{p))-G{p)>-Gj, 
p e W2 , d{p, Tf > Cie =^ G{F{p)) - G{p) > l/C; 
p e Wa \ W2 =^ G{F{p)) - G{p) > Gs log(l/e) . 



In addition we have 



(4.19) 
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for some constants Cg and Ni . 

Eventually we will apply this construction with the small parameter 

_ ^ 

In particular, the condition (4.19) shows that expG is an order function in the sense of 
(3.18) and (3.20). 

Proof of Proposition 4-6: The first three lines of (4.18) are obvious, since x = 1 on VV2 U 
F(W2), and go o F — go > 0. To check the fourth line, we notice that outside VV2, we have 

G{F{p))-G{p) = 0{\og{l/e)) + C,\og{l/e){gooF{p)-go{p)) > (9(log(l/e)) + C5 log(l/e) . 

If C5 is chosen large enough, the right hand side is bounded from below by C8log(l/e) for 
some Cs > 0. 

We then need to check (4.19). We first check it for the function G: we want to show 
with Ci depending on M. Since (p± + Me ~ (p±, this is the same as 



and that follows from the triangle inequality and the properties of (p±: 

^±{p) < c{d{p, nf + e) < c{d{p, + i/i - + 6) 

< G'{^^{p) + \p-p\') = C'i^^ifi) + e{{p - fi)/^ef) 

<2CV±(/^)((P-/^)M)^ 
Inserting the definition of G, (4.20) gives 

\G{p) - Gill) I < C + 2 log((p - //)/yi) . 
The estimate for G is essentially the same: 

\G{p) - G{p)\ < \x{p)G{p) - x(p)G(/i)| + C5log(l/6) \go{p) - go{p)\ 
< C\p - p\ log(l/e) + C log((p - p)/^e) + G 
<G'\og{{p-p)l^e) + G' . 
The last estimate follows from 

xlog J < Clog/^\ +C, 0<x<l. 



□ 
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The function G{x, ^) we have constructed will be used to twist the monodromy operator 
M{z,h), before injecting it inside a Grushin problem. We first recall the structure (and 
strategy) of Grushin problems. 



5. The Grushin problem 

In this section we will construct a well posed Grushin problem for the operator I — M, 
where M — M{z, h) is an abstract hyperbolic open quantum map (or monodromy operator) 
as defined in §2. Wc will treat successively the untruncated operators M, and then the 
operators M truncated by the finite rank projector Hh- The second case applies to the 
monodromy operators constructed as effective Hamiltonians for open chaotic systems [29] , 
and to the operators constructed in §6 to deal with obstacle scattering. 



5.1. Refresher on Grushin problems. We recall some linear algebra facts related to the 
Schur complement formula. For any invertible square matrix decomposed into 4 blocks, we 
have 



a b 


-1 


a 




c d 






S 



a ^ — a 



provided that S ^ exists. As reviewed in [44] this formula can be applied to Grushin 
problems 

R. 







+ ) 



where P is the operator under investigation and R± are suitably chosen. When this matrix 
of operators is invertible we say that the Grushin problem is well posed. If dim'H- = 
dim?{+ < oo, and P — P{z), it is customary to write 



-P{z) 


Rj 


-1 


-E{z) 




R+ 







E_{z) 





and the invertibility of P{z) : Hi is equivalent to the invertibility of the finite dimen- 

sional matrix E_^{z). For this reason, the latter will be called an effective Hamiltonian. 

This connection is made more precise by the following standard result [44, Proposition 
4.1]: 

Proposition 5.1. Suppose that P = P{z) is a family of Fredholm operators depending 
holomorphically on z & Q,, where (Z C is a simply connected open set. Suppose also that 
the operators R± = R±{z) are of finite rank, depend holomorphically on z & n, and the 
corresponding Grushin problem is well posed for z E Q. Then for any smooth positively 
oriented 7 = dT, T 1^ fl, on which P{z)~^ exists, the operator J dzP{z)P{z)~^dz is of 
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trace class and we have 



(5.1) 27ri 



— tr / P{z)-^d,Piz)dz = —iY I E_+(z)-^d,E_+(z) dz 

= #{zer:det£;_+(z) = o}, 

where the zeros are counted according to their multiplicities. 



5.2. A well posed Grushin problem. Let Y d \J-^.^^ M'^, U (E T*Y, F C U x U, he 
a hyperbolic Lagrangian relation, and M — M{z,h) G Io+{Y x Y, F') be an associated 
hyperbolic quantum monodromy operator as in Definition 2.1. In this section it will be 
more convenient to see M as a Fourier integral operator acting on L'^{M.'^)'^ . Let G be the 
escape function constructed in Proposition 4.6, and G^{x,hD) the corresponding pseudo- 
differential operator. 

We will construct a well posed Grushin problem for the operator P{z) ~ I — Mtaiz), 
where 

(5.2) MtG{z) e-*«"'(-''^^) M{z) e*«"'(-.'^^) , t>0. 

For this aim we will need a finite dimensional subspace of L^(]R'')"^ microlocally covering 
an {h/h)2 neighbourhood of the trapped set T. We will construct that subspace by using 
an auxiliary pseudodifferential operator. 

Proposition 5.2. Let F T'*R'^ be a compact set. For the order function 

(5.3) m(x,0 = h/h + d{{x,0,T)\ 
there exists q G 5*1(777.), 50 that 

(5.4) q(x, ~ m(x, , d^q = 0{q'-\°'\/') , 

and such that, for h small enough, the operator Q =^ q^ix, HD) satisfies Q — Q* > h/2h. 

Proof. Let ip be as in Lemma 4.2 with e = h/h. We will take q = Lp, Q = Op^{q). From 
the reality of q, this operator is symmetric on L^(]R''). The estimates (5.4) are automatic, 
as well as the uniform bound q{p) > h/h. Taking into account the compactness of F, the 
estimates (4.3) easily imply that q e S'i(r77). 

To prove the lower bound on Q, we use the rescaling (3.11): 

m = {h/h)^{{h/h)'-^p), V = {h/K)-^V. 

Our aim is to show that Op|'((^) > 1/2 ioi h small enough. This is a form of Garding 
inequality, but for an unbounded symbol. To prove it, we draw from (4.3) 



(P[P2) 1 + d{p2, F)2 
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which shows that is an order function in the sense of (3.13), and (p G S{(p). Similarly, 
the uniform bound <^ > 1 implies that 

(^_A)-ie5(l/^), A<^, 

and hence for h small enough, Op~'(<^) — A is invertible on L^(R'^), uniformly for A < 1/2 
As a result, for h small enough we have the bound 

Opf(^)>l/2. 

RecaUing that 

for h small enough we obtain the stated lower bound on Q = Op]^(g) = Op^((^). □ 

The above construction can be straightforwardly extended to the case where F = T (s 
lA (E (T*]R'^)"': the symbol q — (gj)j=i,...,j is then a vector of symbols qj G S'i(m), and 
similarly Q = (Qj)j=i,...,J is an operator on L^(]R'^)"'. 

Let ^ 1 be a constant to be chosen later. We define the following finite dimensional 
Hilbert space and the corresponding orthogonal projection: 

(5.5) V'^^V,, v/=' 1q^^^,^-,L\R') , Uy diag{lQ^^^,^-,) : L\R'y . 

Due to the eUipticity of Q away from T, see (5.3), we have 

dim V < oo . 

Lemma 5.4 below will give a more precise statement. 

The operators R± to inject in the Grushin problem are then defined as follows: 

(5.6) R+ = Uv : L\Ry — >V, R_ : V ^ L^R^ . 

Before stating the result about the well posed Grushin problem for the operator Mto, 
we prove a crucial lemma based on the analysis in previous sections: 

Lemma 5.3. Let Mta{z) be given by (5.2), z G fl{h), where fl{h) is given in (2.5). Then 
for Hy given above, and any e > 0, there exists t = t{e), sufficiently large, and h = h{t,e), 
sufficiently small (independently of h ^ 0) so that, provided Nq from (2.4) and Cq from 
(5.9) satisfy Nq > ACet, then for h > small enough one has 

(5.7) ||(/ - ny)MtG!||i2(]Rd)j^£,2(Rd)j <£, ||MtG(/ - ny)||^2(]j<i)j^£,2(Rd)j <£. 

Proof. Since M{z) is uniformly bounded for z G Q{h) - see (2.6) - we will work with a fixed 
z and normalize ||M(2;)||i2^i2 to be 1. That is done purely for notational convenience. 
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First, we can choose ijj e C^{[0, 3X/4), [0, 1]), V't[o,ftr/2]= 1, so that 
Uv ij{hQ/h) = ij{hQ/h) , ij{hQ/h) e ^ 1 , 

and hence, 

\\MtG{I - Tlv)\W^L^ = \\MtG{I - HhQ/h)){I - Uv)\\l^-,l^ 

< WMtcil-i^ChQ/hmL^^L^. 

All we need to prove then is (5.7) with 11^ replaced by the smooth cutoff ijj{hQ/h), which 
now puts the problem in the setting of §3.4. 

Before applying the weights, we split M — MAm + M(l — Am), using the cutoff Am of 
Definition 2.1. We then apply Proposition 3.14 to the Fourier integral operator T — MAm, 
with the function oq = 1 and replacing G with —tG. We then get 

M,G = {MAM)tG + (M(7 - AM))tG = MAm e-*^^*^)^^^''^^) e*«"'(^'''^) 

(5.8) +Mi/i^(i-'^)^^$i(e*(«-^*«)) 

+ Oi2^,.2(/i^o-^^«*), 

where Mi e Io+{Y x Y,F'), WFh{Mi) C F(suppaM) x suppaM- 

The error term O L'^^L2{h^°~^^^^) corresponds to ||(-/W'(/ — AM))tG||, it comes from (2.4) 
together with the bound 

(5.9) e^tG-i-m ^ c»^,_^^,(/,-*C6(i+o(/.))) ^ C»^,_^^,(/i-2tC6) ^ I small enough, 

due to the first property in (4.18). 

By contrast, the second line in (4.18) shows that exp(-t(F*G'-G)(p)) < e**^^ for p e W3. 
We have assumed that D suppaj^: this implies that for any t > 0, we have 

||AMe-*(^*«)'"e*«nU^-.L^ \\MiOp^{e'^^-^*^^)U.^,^ < C e'^^' uniformly in 

This is a crucial application of Proposition 3.5 and the properties of the escape function. 
It remains to estimate the norm of 

MAm e-*(^'^)"' e*^™ (/ - ^{hQ/h)) = M bt{x, hD) , he Si (1) , 

b, = aM e-*(^*^-^) (1 - ij{hq/h)) + hSi{l), 

where g is as in (5.4) and qm is the symbol of Am- 
Fixing £ > 0, we first choose t large enough so that 

e-*/^i < e/A 

where Ci is the constant appearing in (4.18). 
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At this point, we can select the constant K > 1 in the definition (5.5). In view of the 
estimates (5.4), we choose it large enough, so that 

q{p) > ^{h/h) ^ d{p,rf > 2C, (h/h) , 

(5.10) penR{F), qoF{p)>^{h/h) ^ d{p,r)^ > 2Ci{h/h) . 

As a consequence, all points p e supp(l — ip{hq/h)) satisfy d{p,Ty > 2Ci{h/h), and 
therefore 

Vp G m, e-*(^*^-^)('')(l - ij{hq{p)/h)) < e-'l^^ < e/4. 
It then follows, see Lemma 3.4, that 

\\Mbr{x,hD)\\L2^L^ <^ + C{t)h. 
Altogether, we have obtained 

\\MtGiI-Ilv)\\L^^L^ < ||M6i(x,/lD)|U2_^2 + O(/ii/ii) + O(/i^0-4^«*) 

< s/2 + Ot{h) + 0{h'^U) + C»(/i^°-^^«*) . 

The assumption A^o > ^C^t ensures that, once we take h < ho{t,e) and take h > small 
enough, the above right hand side is < e. 

A similar proof provides the estimate for (J — Ilv)MtG, replacing Ily by il){hQ /h). using 
Proposition 3.14 to bring ip{hQ/h) to the right of M, and using the assumption (5.10) to 
bound from above (1 — ip{hq o F/h))e~^^^*^~'^\ □ 

The invertibility of the Grushin problem is now a matter of linear algebra: 

Theorem 2. Suppose that M — M{z) is a hyperbolic quantum monodromy operator in 
the sense of Definition 2.1, and Mia the conjugated operator (5.2). We use the auxiliary 
operators of (5.6) to define the Grushin problem 

(5.11) MtG=(^^^^''' : L^W^y ®V ^ L^W^y ®V , 

If t is large enough, h is small enough, and Nq from (2.4), Cq from (5.9) satisfy Nq > 
AC^t, then for h small enough the above Grushin problem is invertible. Its inverse, Etc, is 
uniformly bounded as h ^ 0, and the effective Hamiltonian reads 

oo 

(5.12) E_+ = -{Iv- HyMta) + ^ n^MtG[(/ - Ilv)MtG]' : V ^ V , 

k=l 

where the convergence of the series is guaranteed by (5.7). 
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Proof. We first construct an approximate inverse, 



po 4?? ( ^ ~ R- 



for which we check that 



1 ON W CO _ r ( Mtoil - Uv) (I - Uv)MtG \ 



The theorem now follows from Lemma 5.3 and the Neumann series inversion: 

I + R {I + R){I -Uv)MtG 
Iv 



) , R''^'J2^M,G{I-Ily)r. 
^ k=l 



We finally show that the operator UyMfG, and thus the whole inverse StG, is bounded 
uniformly in h. Consider a cutoff ipi G C^{T*Y) supported inside a small neighbourhood 
of T, equal to unity in a smaller neighbourhood of T. In particular, using the notations 
of Proposition 4.6 we assume that suppF*'0i W2. Since Uy is microlocahzed in a 
semiclassically thin neighbourhood of T, we have 

Uvt/^rix, hD) = ny + e»i2_i2(/i°°) . 

We are then lead to estimate the norm of the operator t/j'^MfG- Using the decomposition 
(5.8) and the fact that {F*G — G){p) > — C7 for p e >V2, we obtain for h, h small enough: 

(5.14) yt > 0, llV'r MtG\\L^-,L^ < e^^^* =^ \\Uv MtGh^-^L^ < C e^^^* . 

□ 

This achieves the reduction of the monodromy operator M{z, h) to the finite rank oper- 
ator E_j^{z). In the next section we perform the same task, starting from a monodromy 

operator M{z, h) which is already of finite rank. 

5.3. Quantum monodromy operators acting on finite dimensional spaces. So far 

we have been considering quantum maps or monodromy operators given by smooth h- 
Fourier integral operators of infinite rank. The quantum monodromy operator constructed 
in [29] and providing an effective Hamiltonian for operators {i/h)P — z, z E D{0,R), was 
given by the restriction of such a Fourier integral operator to a finite dimensional space 
W C L'^{M.^y microlocahzed on some bounded neighbourhood of the trapped set. 

^^^^^ WcL^Wy, dimW^<oo, Uw : L\Ry , 

Mw = liwMIiw + Ow^w{h^') , 
where M G Iq+{Y x Y^F') is a smooth Fourier integral operator. Compared with the 
notations in Definition 2.1, we have 11^ = II/j and M\y = M + Ow^wih^'^)- 
Let us now consider the Grushin problem for {Iw — Mw)- 
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Theorem 3. Suppose that is given by (5.15), with M a monodromy operator as defined 
in Definition 2.1. The space V, and the auxiliary operators R± are as in Theorem 2. We 
construct a weight G as in Prop. 4-6, and such that 11^ = / near suppG. 

If t is large enough, h is small enough, and Nq satisfies Nq > ACgt, with Cq from (5.9), 
then the operator 

_ ( Iw-Mw n,f.e*G™(-''^^)i?_ \ 

MtG = \ \ : W ®V ^ W ®V , 

is invertible, with the inverse 

V E_+ J 

Furthermore, the effective Hamiltonian is uniformly bounded: 

\\E_+\\v^v = 0,{1) . 

Proof. We start by proving three estimates showing that the projector Uw does not interfere 
too much with the Grushin problem. Firstly, in the Definition 2.1 we have assumed that 
Uw is equal to the identity, microlocally near the support of gm- this has for consequence 
the estimate 

(5.16) Am = UwAm + OL^^L^h"^) = Am^w + OL^^LAh"^) . 

Secondly, let us notice that in Proposition 4.6 we required the weight to satisfy supp au ^ 
>V3 suppG. Since we also know that = / some neighbourhood W4 of suppoM, it 
is indeed possible construct the weight G such that W3 suppG W4. This has for 
consequence the estimate 

(5.17) > 0, e-*^'"(^''^^) Hw e*^"'(^''^^) = n^. + Ol2->l2(/i°°) ■ 

(the fact that we are deahng with 5*1 symbol classes does not affect the result, see for 
instance [10, Theorem 4.24]). Thirdly, the definition of the subspace V in (5.5) shows that 
the projector riy is microlocalizcd inside an h^^^ neighbourhood of T, while = / in a 
fixed neighbourhood. This induces the estimate 

(5.18) nvnvK = ny + Oi2^i2(/t~). 

We now want to solve the problem 

^ u \ _ f V 
U_ ) ~ [ v+ 



MtG 

We first consider the approximate solution 



^'\_co fv\ ^0 def /e*«™ \ . fe-^^'- 
^0 1 - ^ta I , ^tG-\ Q ^tG I Q 
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where Eta is the inverse of the Grushin problem in Theorem 2. In particular, 

The estimate (5.16) implies that 

liwMYiw = M + OL2^L^{h^') , 
and hence {v = Uwv as v is assumed to be in W), 

(5.19) {Iw - Mw)Ilwu'' + nvv^e*^™(^''^^)ii° + Ow{h''°\\u''\\L2) . 

Since i?+e^*^™(^''*^)ii° = v+, the definition of R+ = Hy and the estimates (5.17), (5.18) 
imply that 

tG'^{x,hD)-) p-tG'^(x,hD)^0 



Since Eta is uniformly bounded, we obtain the bound 

< C{h-'^^^*\\v\\w + \\v+\\v) 

<C{h-^''''\\v\\w + h~^''''\\v+\\v). 
This bound, together with (5.19) and (5.20), gives 

~ /n.yMn _ (Iw + 0(/i^°-^^«*) C(/i^°-2f^«*) \ / \ 

^ J - V o(/i^°-2^«*) iv + 0(^^°) J v^+y ■ 

The assumption Nq > ACet implies that the operator on the right hand side can be inverted 
for h small enough, and proves the existence of 



\ ivj *^ 1^ e»(/i^o-2c^6t) + c»(/,iVo) J • 



Prom this expression, we deduce the estimate for ||£^tG||, as well as the uniform boundedness 
of 

(5.21) = + e»^^y(/i^°-^^«*) . 

□ 

Remarks. 1) The projector is typically obtained by taking = drngO^wj), ^w,j = 
^Qoj<c where Qqj = Op^{qoj) G \1'(M'^) is elliptic. This way, Uw is a microlocal projector 
associated with the compact region 

Then the estimates (5.16), (5.17) hold if suppaM ^ suppG ^ Wq. 
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2) The requirement that A^o > 4(76^, where the constant Cq depends on G, and t — t{e) 
has to be chosen large enough, seems awkward in the abstract setting {Nq is the power 
appearing in (2.4)). In practice, when constructing the monodromy operator M we can 
take Nq arbitrary large, independently of the weight G (see [29], or the application presented 
in §6). 



5.4. Upper bounds on the number of resonances. Let us first recall the definition of 
the box or Minkowski dimension of a compact subset F M*^: 

(5.22) dimM F = 2/xo =^ k - sup{7 : limsup e"^ vol({p e R'^ : d{p, F) < e}) < oo} . 

e-5-O 

The set F is said to be of pure dimension if 

limsupe-'=+2'*° vol({p e R*^ : d{p, F) < e}) < oo . 

In other words, for e small 

vol({p e R'^ : d{p, F) < e}) < Ce*^""'^ , p > /lo , 

with p replaceable by po when F is of pure dimension. 

In the case of a compact set F = UF^- U^^^^R*^, its Minkowski dimension is simply 

(5.23) dimM F = max dimMFj. 

j=i,...,j 

The following lemma expresses the intuitive idea that a domain in T*R'^ of symplectic 
volume V can support at most h"'^ v quantum states. 

Lemma 5.4. Let F be a compact subset o/T*R'^, of Minkowski dimension 2po. Using the 
operator Q constructed in Proposition 5.2, we take X ^ 1 and define the subspace 



Then, for any p > po there exists G — G^, such that 
(5.24) dimV <Gh-'^ 



h 

When F is of pure dimension we can take p — po in (5.24). 

Proof. Since the order function m(p) ^ oo as |p| ^ oo and Q G ^'i(m), the selfadjoint 
operator Q has a discrete spectrum, hence V is finite dimensional, and 

dim V = ii^{X < Kh/h : A G Spec(g)} . 



44 S. NONNENMACHER, J. SJOSTRAND, AND M. ZWORSKI 

The usual min-max arguments — see for instance [39] or [10, Theorem C.ll] — show- 
that dim V < N ii there exists S > and an operator A of rank less than or equal to N, 
such that 

(5.25) {Qu,u) + Re{Au,u) > {Kh/h + 5)\\u\W ueC^{R'^). 

To construct A, take a = i/^ihq/h), where i/j e C^°°(R), i/^it) = 1 for \t\ < 3K and = 
for \t\ > AK. At the symbolic level, a E Si and a{p) = 1 in the region where q{p) < 3Kh/h. 
Taking into account the fact that q > h/h everywhere, we have 

q(p) + 2Kha{p) /h > {2K + l)h/h, p E T*R'^ . 

The arguments presented in the proof of Proposition 5.2 show that, at the operator level, 
we have for h small enough 

(5.26) Q + Ao>2Kh/h, for Ao = 2Kha'^{x,hD)/h. 

This inequality obviously implies (5.25), with A replaced by the (selfadjoint) operator Aq. 
Our task is thus to replace in (5.25) by a finite rank operator. We do that as in [45, 
Proposition 5.10], by using a locally finite open covering of a neighbourhood of T: 

N(h,h) 

Wh^h^^lP ■■ d{p,ry<m/h}c [j Uk, diam(C/fe) < (/i//i)^. 

fe=i 

The definition of the box dimension implies that we can choose this covering to be of 
cardinalinty 

(5.27) N{h;h)<CKAhlhr, 
for any p, > ^q, and for = /xq if F is of pure dimension. 

To the cover {Uk}, we associate a partition of unity on Wf^j^, 

N{h,h) 
k=l 

where all scminorms arc assumed to be uniform with respect to k. The condition on the 
support of 'Xp in the definition of a and the pseudo differential calculus in Lemma 3.2 show 
that 

N{h,h) 

(5.28) (^- E Xki^,hD))a'"{x,hD) e h^Si . 

k=l 

Hence it suffices to show that for each k — 1, . . . , N{h, h), there exists an operator such 
that 

Xk{x, hD)a'"{x, hD) - Rk e h'^^i , rank(i?fe) < Ch"^ , 
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with C and the imphed constants independent of k. We can assume that, for some p'^ — 
Then consider the harmonic oscillator shifted to the point p*^: 

i=l 

If ipo e C^iR), M^) = 1 for W < 2, Mi) = for \t\ > 3, then MhH^/h) e ^i, and 

(5.29) MhH''/h)xk{x, hD)a'"{x, hD) - x^^, hD)a'"{x, hD) = TZk, TZk G h°^^i , 

where the implied constants are uniform with respect to k. The properties of the harmonic 
oscillator (sec for instance [10, §6.1]) show that ipolhH^ / h) is a finite rank operator, with 
rank bounded by CJi^'^. Hence for each k we can take 

Rk = MhH''/h)Xk{x, hD)a'^{x, hD) , 

and define 

N{h,li) 

(5.30) A = 2K{h/h) Rk , rank(A) < Cdh~'^N{h, h) . 

k=l 

The remainder operators TZk in (5.29) satisfy, for any M > 0, 

||^jfc^fe'||L2^.L2 , ||^fe'^fe'||L2^L2 < C'Af/i ( ^ 

uniformly for A;, /c' = 1, . . . , N{h^ h) . 

Since the supports of the x^'s form a locally finite partition, each remainder TZk effectively 
interferes with only finitely many other remainders. One can then invoke the Cotlar-Stein 
lemma (see [7, Lemma 7.10]) to show that 

N{h,h) 

II Yl ^kh^-^L^ = oChn . 
fc=i 

Using also (5.28), we obtain 

A^Ao + Oi^2_L2{hh'^). 

Consequently, for h small enough the estimate (5.25) holds with S — h/h. In view of (5.27) 
and (5.30) the bound we have obtained on the rank of A leads to (5.24). □ 

We now consider a monodromy operator as defined in Definition 2.1: 

(5.31) n{h) 3 z ^ M{z,h) e ll{Y xY,F'), 
where the depends on z is holomorphic. 
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The decay assumption (2.6) implies that there exists it!o > such that, for h small 
enough, 

(5.32) z en{h), Rez < -Ro =^ ||M(^, /i)||i2_,i2 < 1/2 . 

Using the analytic Predholm theory (see for instance [44, §2]), this imphes that (/ — 
M{z, h))~^ is mcromorphic in Q{h) with poles of finite rank. The multiplicities of the 
poles are defined by the usual formula: 

(5.33) 27r^ /^^(^) 

7,(^) : t 2 + ee^"^* , te [0,2%), 
see Lemma 6.2 below for the standard justification of taking the trace. 

Theorem 4. Suppose that {M{z,h), z G Q{h)}, is a hyperbolic quantum monodromy op- 
erator, or its truncated version M{z,h), in the sense of Definition 2.1, and that T is the 
trapped set for the associated open relation F. Let 2/^0 be the Minkowski dimension of T , 
as defined by (5.22,5.23), with k = 2d and F = T. 

Then for any Ri > and any ji > fio, there exists -ft'^,/?,,, such that 

(5.34) Yl ^M{z)<K^,R,h-\ h^O. 

zeD{0,B.i) 

When T is of pure dimension we can take n = in the above equation. 

Proof. Let us treat the case of the untruncated monodromy operator M{z,h), the case 
of the truncated one being similar. We apply Theorem 2 to the family M{z,h). Since 
the construction of the Grushin problem (5.11) depends only on the relation F and the 
estimates (2.4), we see that for any radius R > 0, the Grushin problem 



(5.35) 



Mtciz) = M " ^'""^'^ ] : L'i^y Q)V^ L^Ry © V , 



R+ 
Mtciz) e-*«"'(-''^^)M(z)e*«""(-''^^) , 

is invertible for t = t{e) > large enough, and the inverse Stoiz) is holomorphic in 2; e 
D{0,R), uniformly when /i — > 0. Using the standard result (see [44, Proposition 4.1] for 
that, and [44, Proposition 4.2] for a generalization not requiring holomorphy) we obtain 



ruMiz) = inf tr / (/ - MtG{C))~'d^MtG{C)dC 

= inf -^tr / E_40-'d^E_^{0dC. 

Since E_^{() is a matrix with holomorphic coefficients, the right hand side is the multi- 
plicity of the zero of det E_^{Q at z. 
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Once < £ < 1/2 and the parameter t — t{e) has been selected, the decay assumption 
(2.6), together with the norm estimate (5.14), show that there exists a radius it! = R{t) > 
such that, for h < ho, 

z e n{h), Rez< -R/A^ \\Uv MtG{z,h)\\L2^L^ < 1/2. 

Together with the expression (5.12), the bound (5.7) and the assumption e < 1/2, this 
shows that E^+{—R/4:) is invertible, with ||£'_+(— i?/4)~^|| uniformly bounded. 

We may assume that R > 4i?i, where Ri is the radius in the statement of the theorem. 
The bound (5.34) then follows from estimating the number of those zeros in D{0.R/4). 
That in turn follows from Jensen's formula, which has a long tradition in estimating the 
number resonances (see [26]). Namely, for any function f{z) holomorphic in ^ e D{0,R) 
and nonvanishing at 0, the number n(r) of its zeros in D{0,r) (counted with multiplicity) 
satisfies, for any r < R, 

f "Mdx = ^ riog \f{re^')\de - log 1/(0)1 . 
Jo X 27r 

Applying this identity to the function /(C) = det£^__|_(C — R/i), we see that 

> mM(z) < n(R 2) < - — — -- / -^dx < - — — -- / -^dx 
' ^ log(3/2) A/, . - log(3/2) io X 



^ bg(3M /°g|det^-.(.)l -log|det^_,(-i?/4)| 

Since ||£'_+(2;)||y^y is uniformly bounded for z G D{0,R) and the rank of E_^{z) is 
bounded by dim!/. Lemma 5.4 gives 

max log I det E. (z) I < Cq dim ^ < Kh'^ , 

zeD{0,R) 

where is as in (5.34). Also, 

-log|det£;_+(-i?/4)| = log|det^_+(-i?/4)-^| < dim log ||£;_+(-i?/4)-^|| 

< Kh-^ , 

where in the last inequality we used the fact that £■_+(— i?/4)"^ is uniformly bounded. 
This completes the proof of (5.34) in the case of an untruncated monodromy operator. 

In the case of a truncated operator M(z, h) — M\y(z, h) + 0(h^°), we apply Theorem 3 
and get an effective Hamiltonian E_^{z) which is also uniformly bounded. The estimate 
(5.21) provides a uniform estimate for E_^{—R/4)~^, and the rest of the proof is identical. 

□ 
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6. Application to scattering by several convex bodies 

We now apply the abstract formalism to a very concrete setting of several convex obsta- 
cles. This will prove Theorem 1 stated in §1. 

The setting of several convex obstacles has been a very popular testing ground for quan- 
tum chaos since the work of Gaspard and Rice [12] but the fractal nature of the distribution 
of resonances have been missed by the physics community. The optimality of the fractal 
Weyl laws in that setting was tested numerically using semiclassical zeta function (hence 
not in the true quantum regime) in [24] - see Fig. 1. The mathematical developments of 
other aspects of scattering by several convex obstacles can be found in the works Ikawa, 
Gerard, Petkov, Stoyanov, and Burq - see [3], [13], [18], [35], and references given there. 

6.1. Resonances for obstacles with several connected components. We first present 
some general aspects of scattering by several obstacles. This generalizes and simplifies the 
presentation of [13, §6]. 

Let Oj d M", J = 1, ■ ■ ■ , J, be a collection of connected open sets. Ok n Oj — 0, with 
smooth boundaries, dOj. Let 

J 

and let 7 be a natural restriction map: 

^ : H'{n) Hl{dn) , {^u)j ^ju u\9o, , 
where we interpret 7 as a column vector of operators. 

Let Ao : i7^(R") L^(R") be the complex-scaled Laplacian, in the sense of [42, §3], 

with Te n Bcn{0,R) = M" n 5Rn(0,i?), Oj d £K"(0,i?), for aU j. Here we identified the 
functions on and functions on To- 

For z e -D(0, r) we define a semiclassical differential operator 
(6.1) P{z)''^''-{-h'Ae-l)-z, 

with the domain given by either H'^{W^) or H^{Q) fl Hq{Q), P{z) is a Fredholm operator 
and we have two corresponding resolvents: 

Ro{z) : L\W) H^{W), R{z) : L^Q) H^{n)nH^{n). 

Here and below r > can be taken arbitrary and fixed as long as h is small enough. 

The operator Ro{z) is analytic in 2; G D{0, r) (the only problem comes from i/h + z = 0) 
and R{z) is meromorphic with singular terms of finite rank - see [42, Lemma 3.5] and. 
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for a concise discussion from the point of view of boundary layer potentials, [26]. The 
multiplicity of a pole of R{z) is defined by 

(6.2) mR{zo) = -^.ixL^in) I R{z)dz , je{zo) : t ^ zo + ee^""'', te[0,2n), 

and e > is sufficiently small. 

A direct proof of the meromorphic continuation and a reduction to the boundary uses 
Poisson operators associated to individual obstacles Oj: 

{P{z)Hj{z)f){x)^0, xeW'XOj, ^,H,{z)f^f. 
We then define a row vector of operators: 

J 

(6.3) H(z):H'^dn)^L\W), H{z)v = ^Hj{z)vj , 

We note that -fH{z)v e H^/^{dn) is well defined. 

The family of operators, H{z), is in general meromorphic with poles of finite rank - see 
the proof of Lemma 6.1 below. 

In this notation the monodromy operator Ai{z) defined in (1.5) is simply given by 
I - Miz) ^H{z) : Hl{dft) ^ Hl{dn) , 



We first state the following general lemma: 
Lemma 6.1. For 2; e M + i{—l/h, 1/h), the operator 

iI-M{z)r' : Hlidn) ^ Hlidn) 
is meromorphic with poles of finite rank. For Rez < 

\\M{z)\\ < ^ 



h\ Kez\ ' 



and consequently for Re^; < —^/h, with ^sufficiently large, {I — M.{z)) ^ is holomorphic 
and 

(6.5) ll(^--^(-))-^llHi(a.H.i(..)<^' Re.<-7A. 

Proof. Wc first consider Re^; < 0. Let Rj{z) be the resolvent of the Dirichlet realization of 
P{z) on M" \ Oj. 
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Let e C^{neigh.(dOj) be equal to 1 near dOj and have support in a small neigh- 
bourhood of dOj. In particular we can assume that the (signed) distance, d{»,dOj) is 
smooth there. Define the extension operator, ^jTj' — I, 

X{x) exp {-d{x, dOjfil - h'Aeo,)/h') 
= 0{h^ : HlidO,) Hl{neigh{dO,)) . 

We also note that 

7fc = 0{h--^) : HliW" \ Oj) nlidOk) , uniformly in h, 
see Lemma 3.1. 
Then 

(6.6) H,{z)=T^^-R,{z)P{z)T^^, 
and 

jkH,{z) = 6,kl - ikRj{z)P{z)T^ . 
The basic properties of complex scaling [42, §3] show that for Xj ^ ^^^(-^(0, R)) which is 1 
near Oj (hence supported away from the complex scaling region) we have 

^ ' ' =0{l/lmC): L\W'\Oj) L^W'XOj), lmC>0, 

where A^-^^ and are the complex scaled and the usual Dirichlet Laplacians on M" \ Oj. 
Hence, 



^kRj{z)P{z)T^ = 0{l/\hRez\) : i7^/'(M" \ Oj) i/f '(M" \ O^) , Re^ < 0. 



This in turn shows that 

M{z) = 0{l/\hRez\) : Hl^dn) Hl{dn), Re^ < 0, 
and consequently that (6.5) holds. 

We know (see for instance [42, Lemma 3.2]) that Rj{z) is meromorphic in D{0,r), and 
using (6.6) we conclude that so is / — /A{z). Analytic Fredholm theory (see for instance 
[44, §2]) shows that invertibility of / — M.{z) for Rez < —7 guarantees the meromorphy of 
its inverse, with poles of finite rank. □ 

We recall the following standard result which is already behind the definition (6.2): 

Lemma 6.2. Let Hi and H2 be Hilbert spaces and let z ^ ^(-s) ^ ^{Hi^^H^), z 
B{z) e C{H2,Hi), z & D G C, be holomorphic families of bounded operators. Suppose that 
z I— 7- C{z) G C{H2,H2), z & D, is a meromorphic family of bounded operators, with poles 
of finite rank. Then for any smooth closed curve ^ <Z D, the operator 

(f A{z)B{z)C{z)dz , 
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is of trace class and 

(6.8) tvH^ 1 A{z)B{z)C{z)dz = trn^ 1 B{z)C{z)A{z)dz . 

Proof. Without loss of generality we can assume that the winding number of 7 is nonzero 
with respect to only one pole of C{z), zq G D. We can write 

C{z) = C,{z) + f2 



k ' 



where Co{z) is holomorphic near zq, and are finite rank operators. Consequently, 
(6.9) Aiz)Biz)Ciz)dz = ^^l^^g^) dz , 

is a finite rank operator, and 



T \fe=l 
K 



^ tiH, B{z)CkA{z) \ ^ 

where we used the cyclicity of the trace: tr^T = trT'5' when 5" is of trace class, and T is 
bounded. Same calculation as in (6.9) gives (6.8). □ 

The main result of this section is the following 

Proposition 6.3. Suppose that the family of operators, z 1-^ H{z), defined in (6.3) is 
holomorphic for z e D{0,ro), Then the resonances, that is the poles of R{z), agree with 
multiplicities with the poles of I — A4{z): 

(6.10) mnizo) = -—tiL^^Qn) i {I- M{z))-'—M{z)dz , 

J-yeiz) dz 

where 7e(-2o) t ^ zq + ee^'^**, t e [0,27r); for sufficiently small e > 0, and the multiplicity 
'>tT'r{zo) is defined by (6.2). 

Proof. We first recall, for instance from [13, §6], that R{z) can be expressed using the 
inverse of I — M.{z): 

(6.11) R{z) = InRoiz) - lnH{z){I - M{z))~'-fRoiz) , 
where Ro{z) acts on functions L'^{Q) ^ L^(]R") extended by 0. Indeed, 

P {inRof - tnH{I - My'^yRof) = PlnRof = f, in f). 
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The Dirichlet boundary condition is satisfied as 

7 {inRoa - tnH{I - M)~'-fRo) = 7^0 - 7^(7^^)"'7^o = . 

Hence by the uniquenss of the outgoing solution (using complex scaling that is simply the 
Predholm property of P{z)) (6.11) holds. 

Since Ro{z) is holomorphic in z, (6.11) and Lemma 6.2 show that 

(6.12) mniz) = -^^ tr / lnH{z){I - Miz))-'-fRo{z) dz 

We want to compare it to the right hand side in (6.10) and for that we will use the 
following 

Lemma 6.4. With the notation above we have 

(6.13) ^M{z) = -'yRo{z)H{z) + (/ - M{z)) diag {jkRo{z)Hk{z)) . 
dz 

Proof. The definition (6.4) gives {d/dz)M.{z) — —^H'{z), and by differentiating 

P{z)Hk{z)^Q, -fkHk{z)^I, 

we obtain 

P{z)H'^{z) = Hk{z) inK"\C»fe, ^kK{z)^Q. 
Arguing as in the case of (6.11) we obtain 

Hl{z) = l^r.\OuRo{z)Hk{z) - Hk{z)jkRo{z)Hk{z) , 

and hence 

^M{z) = -jRo{z)H{z)+jH{z)dieig{jkR^{z)Hk{z)) , 
which is the same as (6.13). □ 

Lemmas 6.2 and 6.4, and the assumption of holomorphy of H{z) in 1^(0, ro), show that 

-trman) / (/ - M{z))-^^M{z)dz = 

(6.14) 

trL2(Rn) d) H{z){I - M{z))~^jR^){z)dz , 

which is awfully close to (6.12). The difference of the right hand sides of (6.14) and (6.12) 
is equal to 

J 



(6.15) 



^tr / to^H{z){I-M{z))-^-iRo{z)dz = 

j=l As (2) 

Vtr / H{z){I - M{z))-^^R^{z)lo,dz . 
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Now observe that 

tRn\OjRo{z)'^Oj = Hj{z)-fjRo{z)loj, 

which imphes 

7fei?o(^)lo,- = 'ykHj{z)'yjRo{z)loj ■ 

This in turn shows that 

^Roiz)lo, = iH{z)Ti,^Ro{z)lo, = (/ - M{z))TifiRo{z)lo, , 

where tt^ : C"^ — )■ C"^ is the orthogonal projection onto Ccj, Cj denoting the jth canonical 
basis vector in C'^. Hence 

H{z){I - M{z))-^^Ro{z)to^ = H{z)^,^R^{z)to,. 

This expression is holomorphic in z since we assumed that H{z) has no poles in the region of 
interest. That proves that the trace in (6.15) vanishes and completes the proof of (6.10). □ 

6.2. Semiclassical structure of the Poisson operator for convex obstacles. We 

now review the properties of the operator H[z) given in (6.3) where it is given in terms 
of Poisson operators Hj{z) for individual convex obstacles. These properties are derived 
from results on propagation of singularities for diffractive boundary value problems (see 
[17, §24.4] and references given there) and from semiclassical parametrix constructions [13, 
A. II], [48, A.2-A.5]. They are based on ideas going back to Keller, Melrose, and Taylor - 
see [27] and references given there. The main result we need is stated in Proposition 6.7 
below. 

To orient the reader we first present a brief discussion of a model case and then use the 
parametrix to prove the general resuls. 

6.2.1. A model case. We will review this parametrix in a special model case where it is 
given by an explicit formula. Using Melrose's equivalence of glancing hypersurfaces [25] this 
model can be used to analyze the general case but due to the presence of the boundary that 
is quite involved [28, §7.3, Appendix A] (see also [28, Chapter 2] for a concise presentation 
of diffractive geometry). 

The model case (in two dimensions for simplicity) is provided by the Friedlander model 
[11], [17, §21.4]: 

(6.16) Po = {hD.^f - X2 + hD,^ , po = ^l-x2 + ^i, 

with the boundary X2 = 0, the Poisson operator Hq. The surface to which Hqu is restricted 
to can be written as X2 — g{xi). The Hamilton flow of po is explicitely computed to be 

{x, ^ (xi + t,x2- ^2 + {t + 6)^; a, 6 + i) , 

and the trajectories on the energy surface po — tangent to the boundary X2 — correspond 
to ^1 = 0. The bicharacteristic concavity of a region qo{x) > (modelling the concavity of 
W^\dOj) is given by the condition Hp^qo > 0: that is automatically satisfied for qo{x) — X2 
and holds for qQ{x) = g{xi) — X2 if g"{xi) > 2. 
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For V e Cc°°(Ra;i) the problem 

(6.17) Pou{x) = Q, X2>0, u{xi,Q) = v{xi) , 
has an exphcit solution: 

(6.18) nix) = H,v{x) = ^ // , 

where is the Airy function solving >l+(i) — tAj^{t) and having the following asymptotic 
behaviour: 



2,3/2 

TT 2t 463'- t — ). +0O , 



7r-t(-t)-3ei^(-*)'^'+^f t^-oo. 

Different asymptotic behaviours corresponds to different classical regions: 

^1 < , hyperbolic region: trajectories transversal to the boundary, 
^1 = 0, glancing region: trajectories tangent to the boundary, 
^1 > , elliptic region: trajectories disjoint from the boundary. 

If V is microlocally concentrated in the hyperbolic region, WF/i(v) ^ {^i < 0}, then 



where i^[x,^^i) = |(— (— 'Ci)^ + + •'^2)^) + Xi^i. That means that Hq is microlocally an 
/i-Fourier integral operator in the hyperbolic region, with the canonical relation given by 

Cq"^^ {{{xi,X2]d^^ip,d^^ip),{d^^ip,^i)) , < 0} 

= {((xi,X2;ei, (-6 +2^2)^), (2:1 -(-6 + 2:2)^ + (-6)^6)), 6 < 0} 

= {((xi, (xi - yi + (-Ci)^)' + -yi + (-6)^), (1/1,6)) , 6 < 0} . 

This corresponds to outward trajectories starting at (?/i, 0) and explains why this choice of 
an Airy function gives the outgoing solution to (6.17). 

The propagation of semiclassical wave front sets is given by taking the closure of this 
relation which is smooth for ^ < only: 

(6.19) "^YhiHov) n {X2 > 0} = Co(WFfc(t;) n {6 < 0}) n {X2 > 0} . 

This can be proved using (6.18). Strictly speaking the wave front set on the left hand side 
is defined only in the region {x2 > 0} because of the presence of the bounday X2 = 0. 

We now consider 'yiu{xi) =^ u{xi, g{xi)) and (putting x = Xi), 

(6.20) M) ^ ^ // ^i^3^ei<'-"„fe)d. . 
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When acting on functions with MVFh{v) {(?/, ^) : rj < 0}, we can again use asymptotics 
of and that shows that, microlocally for 77 < — c < 0, is an /i-Fourier integral 

operator with a canonical relation with a fold [17, §21. 4], [27, §4]: 

Bo '^^ {{x,rj + g'{x){-rj + g{x))^; X + {-rj)^ - {-rj + g{x))^ ,rj) , rj < 0} . 

This means that the map / : i3o — )■ T*]R, the projection on the second factor, f{x, ^, y, rj) = 
{y,r)), at every point at which 

g'{x) = 2{-7] + g{x))^ , 

satisfies dimker/' = dimcoker/' = 1, and has a non-zero Hessian, 

ker f 3X^ {fx, X) e coker /' . 

The Hessian condition is equivalent to g"{x) 7^ 2 which is satisfied as we assumed that 
g"{x) > 2. This corresponds to the tangency of the trajectory X2 — {xi — y + {—r}Y^'^Y + r], 
to the boundary X2 = g{xi). Using either an explicit calculation or general results on folds 
(see [17, Theorem 21.4.2 and Appendix C.4]) we can write 

(6.21) Bo^B+UBo, 

where B^ correspond to trajectories entering (+) and leaving (— ) X2 > g{xi). Using (6.20) 
one can show a propagation result similar to (6.19): 

(6.22) WFhi^.Hov) = Bo{WFh(v) n {6 < 0}) . 



6.2.2. Arbitrary convex obstacle. To handle the general case we introduce the following 
notation 

S*9o,^^ = {{x, e T*W : xedO,, lei = 1} , 
S*dOk^{(y,rj)eT*dOj, M = 1} , 
B*dO,^{{y,r})eT*dOk : < 1} , : S^o.^ ^ B*dO, . 

where | • | is the induced Euclidean metric, and iTk is the orthogonal projection. 

We first recall a result showing that when considering ^kHj{z) we can restrict our atten- 
tion to a neighbourhood of B*dOk x B*dOj: 

Lemma 6.5. Suppose that dO^, £ — j, k, Oj nOk — 0, are smooth and that Oj is strictly 
convex. If xe e S'^'^{T*dOi) satisfy xi=^ near B*dOe, £ = j, k, then for z e D{0, r), 

{l-xJhkHj{z) = 0{h^) : L\dO,)^C^{dOk). 
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Proof. The first estimate follows from the parametrix construction in the elliptic region - 
see [13, Proposition A.II.9] and [48, §A.-A.5]. Prom [48, (A.24)-(A.26)] we see that for 

i;{x)H^{z){l-xJ) = OL^^co^{h^). 
and the first estimate in (6.23) follows. As a consequence we can extend '-)kHj{z) : 
Hl{dOj) Hl{dOk) to 

(6.24) 7fciJ,(z) : L\dO,) — > L^dOk) . 

Once ^ikHj{z) is defined on L^, the second part of (6.23) follows from the fact that 
(— /i^A — l)Hj{z)v = and hence WFh{ipHj{z)) C {|^| = 1}. We simply apply Lemma 
3.1. □ 

The next proposition establishes boundedness properties: 

Proposition 6.6. Suppose that dOi, i — j,k are smooth and that Oj is strictly convex. 
Then 

(6.25) jkHj = 0{l/h) : L^dOj) L\dOk) . 

We cannot quote the results of [13] directly since for similar estimates in [13, A.II.l] the 
Lax-Phillips odd dimensional theory is invoked. As in [13] our proof is based on propagation 
of singularities for the time dependent problem, but it uses the more flexible method due 
to Vainberg [52] . 

Proof. Let H'KdOe) denote semiclassical Sobolev spaces defined in (3.5). 

As in the proof of Lemma 6.1 we will use the resolvent of the Dirichlet Laplacian on 
\ Oj, denoted below by Aj. We also use the extension operator Tj* defined there. 

Following (6.6) we write 

^kH,=5,,-^kR,iz)Piz)T^ : HlidO,)^HlidO,), 

and we need to show that for z G [—Co \og{l/h), Rq] + i[—RQ, Rq] the bound is 0{l/h). In 
view of the discussion above that means showing that for G C^{neigh.{dO£)), 

(6.26) VkRj{z)v, = 0{1) : L2(M" \ Oj) Hl{R^) . 
We recall that LpkRj{z)ipj = (pkPj{z)^^^Pi, where 

P,(z) = {{t/h){-h'A^,g -l)-z) = 0{l/h) : Hl{W^ \ O,) — > L\W^ \ O,) , 

and Aj^e is the complex scaled Dirichlet Laplacian on \ Oj - see (6.1). The rescaling 
involved in the definition shows that we need 

(6.27) (^^((-/i^A,-, - 1) - hzjiY^^j = 0{l/h) : L^W' \ Oj) i/^^M") , 
which follows from 

(6.28) ipk{-Aj-e)-'Vj = OL^^L<V\C\), ImC>-C, ReC>C, 
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where, as in (6.7) we have replaced A^^^ by the unsealed operator. 

To establish (6.28) we use Vainberg's theory as presented in [40, Section 3] and [50, 
Section 3]. For that we need results about the wave propagator. 

Let Uj{t) =^ sint^-Aj/^-Aj be the Dirichlet wave propagator. Then for x e 
(6.29) xUj{t)x : L\W^ \ O,) C^{W^ \ O,) , t>T^, 

where C°° denotes the space of extendable smooth functions. This follows from the singu- 
larities propagate along reflected and glancing rays and that there are no trapped rays in 
the case of one convex obstacle - see [17, §24.4]. 

Applying [40, (3.35)] or [50, Proposition 3.1] to (6.29) gives (6.28), and thus completes 
theproof of (6.25). □ 

We now set up some notations concerning the symplectic relations associated with our 
obstacle system. For i ^ j we now define the (open) symplectic relations R^, analogues of 
the relations Bq (6.21) in the Friedlander model. For x e dOj denote by lyjix) the outward 
pointing normal vector to dOj at x. Then 

{p,p')eBfjCB*dOiXB*dOj 
3t>0,^e §"-^ , xedOj, x + t^e dOi , {iyj{x),0 > , 

±{ui{x + ^0, < , 7r,(x, = P' , + t^:O^P: 

Notice that B^j is equal to the biUiard relation Fij defined in (1.6). 
?± 



The relations Bf, are singular at their boundaries 



(6.31) dBfj = n {B*dOi X S*dOj U S*dOi x B*dOj) , 

which corresponds to the glancing rays on Oj or Oi. We will often use the closures of 

these relations, Bfj, which include the glancing rays. The inverse relations (C* {{p,p') : 
{p',p) G C}) are obtained by reversing the momenta: 

(6.32) {B±Y^JoB±oJ, J{y,v)''^{y,-V), {y,v) ^ B*dO . 
If we define 

J 

(6.33) U = neigh{B*dO) = \_\ neigh{B*dOj) , 

i=i 

we are in the dynamical setting of §2. Compared with §2, we define the arrival and departure 
sets from the closure of the relation Fij: 

def T^X/ ox/-> \ ^ T-,*r^yy-\ def 



Aij = B±{B*dOj) C B*dOi , Dij = {B±y{B*dOi) C B*dOj . 




Figure 6. (after [32]) Partial boundary phase space B*Oi for the sym- 
metric three disk scattering shown in Fig. 3. We show the boudaries of the 
departure {Dn, full lines), arrival {Aij, dashed lines) and shadow arrival {A'^j, 
long dashed) sets. 

From (6.32) we check that 

Dij = J {Aj^ . 
Besides, the shadow arrival sets are given by 

Also, let us call 

(6.34) At^ = \jAi\ D,'='[jD,,. 

The subsets of glancing rays are denoted by 

Af = Ai n S*dOi = A- n S*dO,, Df = n S*dOi . 

With this notation we can state the most important result of this section: 

Proposition 6.7. For i ^ j, let Hj{z) and ji be as in ^6.1, and assume that dO^, k 
are strictly convex. For any tempered v e LP'{Oj), we have 

(6.35) W¥n{liH^{z)v) = (b± U B;^ (WFfe(^;) n B*dO^) , 
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uniformly for 



(6.36) 



dcf 

'0 — 



[-Co\og{l/h),Ro]+t[-Ro,Ro], Co,Ro>0 fixed. 



ifQk^^' 
(6.37) 



0,— oo 
h 



{dOk), k = i,j, satisfy 



then 



(6.38) 



Because of the assumptions on Qk 's, only compact subsets of the open relations Bf^ are 
involved in the definition of the classes 7°. 

We also have, for some r > and z in the above domain, the norm estimate 



Although we will never have to use any detailed analysis near the glancing points (that 
is, points where the trajectories are tangent to the boundary) it is essential that we know 
(6.35) and that requires the analysis of difTractive effects. In particular, we have to know 
that there will not be any propagation along the boundary. 

Proof. For z e VLq, [13, Theorem A. II. 12] gives the wave front set properties of Hj[z). In 
particular it implies that for ipi- G C^(neigh((9Cfc)), ipk = Q near O^, 



This and Lemma 3.1 immediately give (6.35). 

The conditions on Q^'s appearing in (6.38) mean that we are cutting off the contributions 
of rays satisfying {uji^x),^) = 0, that is with 7rj(^) G S*dOj (glancing rays on Oj), as well 
as the contributions of the rays (z/j(a; + t^),^) =0 (glancing rays on Oi). This means that 
the contributions to Qi^iHj{z)Qj only come from the interior of the hyperbolic regions on 
the right, {B*dOj)°, and on the left, {B*dOi)°. The description of Hj{z) in the hyperbolic 
region given in [13, Proposition A. II. 3] and [48, §A.2] shows that it is a sum of zeroth 
order Fourier integral operators associated to the relations Bfy For Rez < the forward 
solution of the eikonal equation gives the exponential decay of (6.39) (where < r < djj, 
the distance between Oj and Oi). As pointed out in [48, §A.2], this decay is valid for z in 
the logarithmic neighbourhood VLq. □ 

6.3. The microlocal billieird ball map. In this section we will show how for several 
strictly convex obstacles satisfying Ikawa's condition (1.1) the operator M.{z) defined in 
§6.1 can be replaced by an operator satisfying the assumptions of §2. This follows the 
outline presented in §1. 



(6.39) 



\\QaiHj{z)Qj\\L^(^do^)^L^{dOi) < C{Ro)exp{TRez) . 



WFh{(pkHj{z)v) = {{x + t^,0 : t>0, x + t^ esnpp^k, 1^1 = 1, 

{x,7ij{0) e WF,(^;) n B*dO, , {i^j{x),0 > 0} . 
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Namely, we now show that invertibihty of / — A4{z) can be reduced to invertibihty 
of / — M{z) where M{z) satisfies the assumptions of §2. That can only be done after 
introducing a microlocal weight function. 

We first consider a general weight. Suppose that 

go e SiT*dO; (0'^ = fl S{T*dO; (C)-^) , 

N>0 

or simply that go e C^{T*dO). Then for a fixed but large T consider 

(6.40) g "^'Tlog (^^^ go , exp{±g'"{x, hD)) e ^l^''\dO) , ^^o = max |^o| • 

We remark that the operators exp(±g'"'(x, hD)) are pseudo-local in the sense that 

(6.41) W¥h{e^3"^'''^°^v) = WYh{v) 

(we see the inclusion from the pseudodifferential nature of exp(±g("'), and the equality from 
their invertibihty). 

For k j, we will write 

(6.42) {M,),,{z) e-^^^'^^) M,,{z) e^^^''^^^ . 

As before, we consider these operators as a matrix acting on L^{dO) — ^j^i L^{dOj). 
Using Proposition 6.6 we see that Ai{z) is bounded by C/h as an operator on L'^{dO). 
uniformly for z & flo (see (6.36)). Since exp{±g'^{x, hD)) = OL2^i^2[h~^'-''°) the conjugated 
operator satisfies Mg{z) = Oi^^L^ih^'^'^^^^^)- 

We want to reduce the invertibihty of / — M.g{z) to that of / — Mg{z), where M{z) is a 
Fourier integral operator. That means eliminating the glancing contributions in M.{z) (see 
(6.38)). 

We start with a simple lemma which shows that Ikawa's condition (1.1) eliminates glanc- 
ing rays and the restrictions to shadows: 

Lemma 6.8. Suppose that for some j ^ i ^ k 

Oi n convex hu\\{Oj U O^) = . 

Then, in the notation of Proposition 6. 7, 

(6.43) ^o^ = 0- 

Proof. Suppose that (6.43) does not hold. Then there exists x e dOj, ^ e S"~^, and 
< ti < t2, such that x + ti^ & Oi, and x -\-t2i G dO}.. But this means that Oi intersects 
the convex hull of dOj and dO^. 

□ 
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The trapped set T was defined using (1.7), where we recall that the relation F — (Fij) 
is also given by Fij — B^j. Lemma 6.8 shows that 

Lemma 6.8 implies that (see Fig. 6) 

(6.44) ^(lf) = 0, so that 5fnlf = 0, and similarly (b^^ {Df) ^ . 

Since the sets Af and Df are closed and disjoint, we can find small neighbourhoods 

(6.45) C/^ neigh(If ) , neigHDf) , C/f n C/^ = , 
so that 

(6.46) ^(t/^) = 0, (fiS)\[/f) = 0- 

We can now formulate a Grushin problem which will produce the desired effective Hamil- 
tonian. For that let Xj,A^Xj,D e C°°{T*Oj;R) satisfy 



XjA fneigh(If 1 ' SUpp Xj,A (E Uf , 
Xj,D tneigh(De)= 1 , SUpp Xj,D ^ Uf . 



(6-47) 



We let Xj,» have the same properties as Xj,» with Xj,» = 1 on suppxj^,. In view of (6.35) 
and (6.46), we have 

Mkj{z) xJ,a{^, hD) = OL2^do,)^c^ido,){h°°) , 
Xlnix, hD) Mji{z) = OL2(^dOi)->c^{do,){h'^) , 

for all k ^ j ^ uniformly for z G VLq. In view of (6.41), and using the notations of (6.42), 
we have the same properties for the conjugated operator: 

, . {Mg)kAz) xIa{x, hD) = OL2^9o,)^c^{ao,)ih'") , 

^ ■ ^ xId{^, hD) {MM^) = OL2^9o,)^c^oo,){hn ■ 

For • — A,D, let Ilj,, be an orthogonal finite rank projection on L'^{dOj) such that 

xl. n„. = n,,. xl. + Ol.^c- {hn = xl. + Ol^^c-^ {h^ , 

Such projections can be found by constructing a real valued function ipj^, G C^{T*dOj) 
satisfying ipj^, = 1 on supp(xj>) and ipj^, = on supp(l - Xj,.). Then Uj^, '^^|;v^{x,hD)>l/2 
provides a desired projection of rank comparable to h 



l-n 
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We need one more orthogonal projector Hf, microlocally projecting in a neighbourhood 
of B*dOj. Precisely, we assume that for some cutoff Xj ^ C^{T*dOj) with Xj — 1 near 
B*dOj U Uf U U^^, this projector satisfies 

(6.50) nf xJ = xJ nf + O^2_coo(/i-) = nf + Or^^^c^hn ■ 
Prom Lemma 6.5 we easily get the bounds 

(6.51) {Mg)kj{z) {I - nf) = C»i2^C-(/i°°) , (/ - nf) {Mg)kj{^) = C»L2^Coo(/l°°) . 

The operator 

(6.52) Pj = n,- ^ + n,- Z5 + / - nf . 

is not a projection but it can be easily modified to yield a projection with desired properties: 

Lemma 6.9. Let 7 be positively oriented curve around — 1, ^ : t ^ 1 + eexp{2Trit), e 
small and fixed. For Pj given by (6.52) define 

Then Ilj is an orthogonal projection satisfying 

(6.53) n,- = Uj,A + ff,-,D + / - nf + Ol^^oo (/i°°) , 

where Hj,,, H'j are the projections in (6.49) and (6.50). 

Proof. The operator Pj is not a projection but it is self-adjoint and satisfies 

(6.54) Pf = P^ + Oi^2^c^{h'^), 

which we check using (6.49), (6.50) and the properties of the functions Xj.»i Xj- nence its 
spectrum is contained in [0, 0(/i°°)] U [1 — 0{h°^), 1 + C(/i°°)]. For e small enough and fixed 
we can take 7 including all spectrum near 1 (the statements implicitly assume that h is 
small enough) . For C e 7 we write 

(c - p^r' = c\i + (c - ly'Pi) [I - c\c - i)-\p] - p,)T" ■ 

The inverse on the right hand side exists in view of (6.54) and satisfies 

(J - c-^(c - i)-\p! - p,)Y' = I + o^'^^cAhn 

uniformly on 7. Inserting these two formulae into the integral defining n^ gives (6.53). □ 

The next lemma provides the property crucial in the construction of the Grushin problem: 

Lemma 6.10. For Uj defined above and any k ^ j ^ i we have 

{Mg)uJ{z)%{Mg),^{z) = 0{h'^) ■ L\dOi)^C°^{dOk), 
uniformly for z E flo. 
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Proof. To simplify the notation we write Mij instead of {M.g)ij{z). 
Using (6.53) and (6.51), we can write 

Mkj fii Mji = Mkj{fij,A + f^j,D)Mji + OL2^c-(/i°°) • 
From (6.49) we use 

U,,A = xIa %,a + 0{K-) , U,,n = fL,,D x7,d + (^(^°°) , 
and hence, using (6.48), we complete the proof. □ 

Now define the following orthogonal projection: 

(6.55) Uh = diag(/ - ff,) : L\dO) L\dO) . 

Since each / — Uj is microlocalized on a compact neighbourhood of B*dOj, it has a rank 
comparable with and so does Uh- 

Using this projection we obtain the main result of this section: 

Theorem 5. Let Uh be given by (6.55) and (6.49), and A4g{z, h) be defined by (6.4) and 
(6.42). 

IfOj are strictly convex and satisfy the Ikawa condition (1.1) then the following Grushin 
problem is well posed for z e I?(0, C): 

where H*^ : Wu ^ L'^{dO). 

The effective Hamiltonian is given by 

E_+{z, h) = - (^Iw, - Uh {Mgiz, h) + R{z, h))lih) , 

where 

(6.56) Mg{z, h) = ^-Mg{z, h)^- e ll^{dO X dO, F') , = diag(l - xIa - xId) , 
with the relation F given by (1.6), Xj^» satisfying (6.47), and 

R{z, h) = CL2(ao)-^c°°(90)(^°°) ■ 
Proof. We first observe that Lemma 6.10 gives 

(6.57) Mg{z, h)Y{hMg{z, h) = Mgiz, hf + OLHdo)~,c^i^ao) (h'^) ■ 

The theorem follows from this. Indeed, (6.57) implies that, in abbreviated notation, 

'I-Mg f{I+{I-Il)Mg){I-U) U'+{I-U)Mgn'' 

n o)\ u{i + Mg{i -n)) -{iw-nMgii) 
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The exact inverse in then obtained by a Neumann series inversion. In view of (6.49) and 
(6.38) we obtain (6.56). □ 



Proof of Theorem 1. To apply Theorem 4 from §5.4 we need to show that we can choose 
g so that Mg{z,h) satisfies the conditions in Definition 2.1. The only assumption that 
needs verification is (2.4). For that we take Qq in (6.40) given in Lemma 4.5 (proved in the 
appendix), with the map being the billiard ball relation, F. 

Egorov's theorem (Proposition 3.10) then shows that (2.4) holds and we can make W as 
close to the trapped set as we wish. □ 

Appendix: Construction of an escape function for an open map F 

In this appendix we prove Lemma 4.5 by explicitly constructing an escape function gg. 
The only assumption we need on F is the fact that the trapped set T ^ D, T A, where 
D, resp. A are the (closed) departure and arrival sets of F. Our construction is inspired 
by similar constructions in [6, Lemma 4.3] and [53]. Here we will independently construct 
functions g± with good escape properties away from the incoming and outgoing tails 7^, 
respectively. 

Let us start with the function g+. We can take V± U, open neighbourhoods of the 
tails 7±, such that 

where W2 is the neighbourhood of T in the statement of Lemma 4.5. We will first construct 
a function g+ e C^{T*Y) such that 

VpeWs, ^+(F(p))-^+(p)>0, 
^ ■ ^ ypeWs\V_, g4F{p))-g4p)>l. 

We will perform a local construction, based on a finite set of points p E D \ V-. Consider 
a compact set W4 such that Ws ^ W4, <^ D. Take any point p e W4 \ V-. We define its 
forward escape time by 

n+ = n+{p) = min{A; > 1, F''{p) ^ W4} ■ 

Since p ^ this time is uniformly bounded from above. Besides, the forward trajectory 
{F'^(p), < k < n+} is a set of mutually different points, with F'^+{p) & A \ VV4. Since 
A \ W4 is relatively open, there exists a small neighbourhood Vp of p, such that the neigh- 
bourhoods F''{Vp), < A; < n+ - 1, are all inside -D \ 71, while F"+(T/p) e A \ W4 (see 
Fig. 7). 
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Figure 7. Sketch of the construction of an escape function (7^,+. The blue 
(resp. pink) circle denotes the neighbourhood W2 (resp. W4) of the trapped 
set inside D, the vertical green rectangle is the neighbourhood V- of the 
incoming tail 71. The three ellipses with patterns indicate the sets F^iVp) 
surrounding F^{p), < k < 2. Here the escape time n^{p) = 2. 



Take a smooth cutoff Xp ^ ^'^iYp, [0, 1]), with Xp = 1 in a smaller neighbourhood V^' d 
of p, and consider its push-forwards 



Xp,k 



def 







I. , 0<k<n^ 
outside F''{Vp), ~ ~ 



The supports of the xp,k, < k < — 1, are all contained in D \ 71, while suppxp,n+ C 
A \ W4. From there, we define 



n_|_ — 1 



the set 1^;+'=' U F\V;)CD, 

k=0 

n+ 

the function Qp j^ = ^^(A; + l)Xp,fc • 

fc=0 

The function Qp j^ is smooth, and on D it satisfies 

rt-i- — 1 

9p,+ °F - gp^+ = XpoF + ^ Xp,k - {n+ + l)Xp,i 



k=0 
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The properties of the supports of the Xp,k imply that 

gp,+ oF- gp,+ > in W4, gp,+ oF- gp^+ > 1 in + n W4 . 

Since W4 \ VI is a compact set, we may extract a finite set of points {pj G W4 \ V^-}j=i,...,j 
such that IJj=i open cover of W4 \ V_. The sum 

J 

def \ ^ 

j=l 

is smooth in U, and satisfies the properties (A.l). Furthermore, for each pj the function 
(7p^ + vanishes near 71, so there exists V!_ <^ V- a neighbourhood of 71 such that all gpj,+, 
and also g+, vanish on V^. 

Applying the same construction in the past direction, we construct a smooth function 
Q- and a neighbourhood K[ d K|. of 7+, such that 

\/peFiVi), ~g-ip)^0, 
VpGF(W3), ~g.iF-\p))-~g4p)>0, 
ypeFiWs\V+), ~g.{F-\p))-~g_{p)>l. 

(notice that the sets F(V^_[), -F(V"+) and W3 have the appropriate properties with respect 
to F~^ : A\-^ D). The function g^ =^ —g^ o F then satisfies 

Vpeyj, ^-(p) = 0, 
(A.2) VpeWs, g-{F{p))-g-{p)>0, 

ypeWs\V+, g^{F{p))-g_{p)>l. 

The function g^ ^= g+ + g_ satisfies conditions in Lemma 4.5, and vanish on Wi V^nV!^. 
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